{\rtf1\windows \deff0{\fonttbl
{\f0\fswiss MS Sans Serif;}{\f1\fmodern Courier New;}{\f2\ftech Symbol;}}\fs20
\page #{\footnote hcAbout}${\footnote \pard{}About this help file}\pard{\fs24\b About this help file}\par\par\pard{}This file was made with the help of {\ul Makertf 3.12b-1}{\v hcMakertf} from the input file recode.texi.\par\par{{START-INFO-DIR-ENTRY\line
* recode: (recode).     Conversion between character sets and surfaces.\line
END-INFO-DIR-ENTRY\par
\pard{}}}\par
{\pard{}This file documents the {\f1{}recode} command, which has the purpose of converting files between various character sets and surfaces.\par
\par
\pard{}Copyright (C) 1990, 93, 94, 96, 97, 98, 99 Free Software Foundation, Inc.\par
\par
\pard{}Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.\par
\par
\pard{}Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.\par
\par
\pard{}Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Foundation.  }\par
\par
{\page\pard Node: {\b Top}, \keepn Next: {\uldb Tutorial}{\v Tutorial}, Prev: {\uldb (dir)}{\v Top @dir.hlp}, Up: {\uldb (dir)}{\v Top @dir.hlp}\tab\tab{\uldb About this help file}{\v hcAbout}\line
K{\footnote K Top}
#{\footnote Top}
${\footnote {\f1{}\pard{}recode}}\par
\pard\pard{\fs24\b {\f1{}recode}}\par
\par
\pard{}This recoding library converts files between various coded character sets and surface encodings.  When this cannot be achieved exactly, it may get rid of the offending characters or fall back on approximations.  The library recognises or produces nearly 150 different character sets and is able to convert files between almost any pair.  Most RFC 1345 character sets are supported.  The {\f1{}recode} program is a handy front-end to the library.\par
\par
\pard{}The current {\f1{}recode} release is 3.5.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Tutorial}{\v Tutorial}\tab Quick Tutorial\par
{\uldb Introduction}{\v Introduction}\tab Terminology and purpose\par
{\uldb Invoking recode}{\v Invoking_recode}\tab How to use this program\par
{\uldb Library}{\v Library}\tab A recoding library\par
{\uldb Universal}{\v Universal}\tab The universal charset\par
{\uldb Tabular}{\v Tabular}\tab Tabular sources (RFC 1345)\par
{\uldb ASCII misc}{\v ASCII_misc}\tab ASCII and some derivatives\par
{\uldb IBM and MS}{\v IBM_and_MS}\tab Some IBM or Microsoft charsets\par
{\uldb CDC}{\v CDC}\tab Charsets for CDC machines\par
{\uldb Micros}{\v Micros}\tab Other micro-computer charsets\par
{\uldb Miscellaneous}{\v Miscellaneous}\tab Various other charsets\par
{\uldb Surfaces}{\v Surfaces}\tab All about surfaces\par
{\uldb Internals}{\v Internals}\tab Internal aspects\par
\par
  { --- The Detailed Node Listing ---\par
\par
  Terminology and purpose\par
\par
 \par
{\uldb Charset overview}{\v Charset_overview}\tab Overview of charsets\par
{\uldb Surface overview}{\v Surface_overview}\tab Overview of surfaces\par
{\uldb Contributing}{\v Contributing}\tab Contributions and bug reports\par
\par
  How to use this program\par
\par
 \par
{\uldb Synopsis}{\v Synopsis}\tab Synopsis of {\f1{}recode} call\par
{\uldb Requests}{\v Requests}\tab The {\i request} parameter\par
{\uldb Listings}{\v Listings}\tab Asking for various lists\par
{\uldb Recoding}{\v Recoding}\tab Controlling how files are recoded\par
{\uldb Reversibility}{\v Reversibility}\tab Reversibility issues\par
{\uldb Sequencing}{\v Sequencing}\tab Selecting sequencing methods\par
{\uldb Mixed}{\v Mixed}\tab Using mixed charset input\par
{\uldb Emacs}{\v Emacs}\tab Using {\f1{}recode} within Emacs\par
\par
  A recoding library\par
\par
 \par
{\uldb Outer level}{\v Outer_level}\tab Outer level functions\par
{\uldb Request level}{\v Request_level}\tab Request level functions\par
{\uldb Task level}{\v Task_level}\tab Task level functions\par
{\uldb Charset level}{\v Charset_level}\tab Charset level functions\par
{\uldb Errors}{\v Errors}\tab Handling errors\par
\par
  The universal charset\par
\par
 \par
{\uldb UCS-2}{\v UCS_2}\tab Universal Character Set, 2 bytes\par
{\uldb UCS-4}{\v UCS_4}\tab Universal Character Set, 4 bytes\par
{\uldb UTF-7}{\v UTF_7}\tab Universal Transformation Format, 7 bits\par
{\uldb UTF-8}{\v UTF_8}\tab Universal Transformation Format, 8 bits\par
{\uldb UTF-16}{\v UTF_16}\tab Universal Transformation Format, 16 bits\par
{\uldb count-characters}{\v count_characters}\tab Frequency count of characters\par
{\uldb dump-with-names}{\v dump_with_names}\tab Fully interpreted UCS dump\par
\par
  ASCII and some derivatives\par
\par
 \par
{\uldb ASCII}{\v ASCII}\tab Usual ASCII\par
{\uldb ISO 8859}{\v ISO_8859}\tab ASCII extended by Latin Alphabets\par
{\uldb ASCII-BS}{\v ASCII_BS}\tab ASCII 7-bits, BS to overstrike\par
{\uldb flat}{\v flat}\tab ASCII without diacritics nor underline\par
\par
  Some IBM or Microsoft charsets\par
\par
 \par
{\uldb EBCDIC}{\v EBCDIC}\tab EBCDIC codes\par
{\uldb IBM-PC}{\v IBM_PC}\tab IBM's PC code\par
{\uldb Icon-QNX}{\v Icon_QNX}\tab Unisys' Icon code\par
\par
  Charsets for CDC machines\par
\par
 \par
{\uldb Display Code}{\v Display_Code}\tab Control Data's Display Code\par
{\uldb CDC-NOS}{\v CDC_NOS}\tab ASCII 6/12 from NOS\par
{\uldb Bang-Bang}{\v Bang_Bang}\tab ASCII ``bang bang''\par
\par
  Other micro-computer charsets\par
\par
 \par
{\uldb Apple-Mac}{\v Apple_Mac}\tab Apple's Macintosh code\par
{\uldb AtariST}{\v AtariST}\tab Atari ST code\par
\par
  Various other charsets\par
\par
 \par
{\uldb HTML}{\v HTML}\tab World Wide Web representations\par
{\uldb LaTeX}{\v LaTeX}\tab LaTeX macro calls\par
{\uldb Texinfo}{\v Texinfo}\tab GNU project documentation files\par
{\uldb African}{\v African}\tab African charsets\par
{\uldb Cyrillic}{\v Cyrillic}\tab Cyrillic charsets\par
{\uldb Texte}{\v Texte}\tab Easy French conventions\par
{\uldb Mule}{\v Mule}\tab Mule as a multiplexed charset\par
\par
  All about surfaces\par
\par
 \par
{\uldb Permutations}{\v Permutations}\tab Permuting groups of bytes\par
{\uldb End lines}{\v End_lines}\tab Representation for end of lines\par
{\uldb MIME}{\v MIME}\tab MIME contents encodings\par
{\uldb Dump}{\v Dump}\tab Interpreted character dumps\par
{\uldb Debugging}{\v Debugging}\tab Artificial data\par
\par
  Internal aspects\par
\par
 \par
{\uldb Main flow}{\v Main_flow}\tab Overall organisation\par
{\uldb New charsets}{\v New_charsets}\tab Adding new charsets\par
{\uldb New surfaces}{\v New_surfaces}\tab Adding new surfaces\par
{\uldb Design}{\v Design}\tab Comments on the library design\par
\par
  }\par
\pard{}}\par
}\page\pard Node: {\b Tutorial}, \keepn Next: {\uldb Introduction}{\v Introduction}, Prev: {\uldb Top}{\v Top}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Tutorial}
#{\footnote Tutorial}
${\footnote \pard{}Quick Tutorial}\par
\pard{\fs24\b Quick Tutorial}\par
\par
\pard{}So, really, you just are in a hurry to use {\f1{}recode}, and do not feel like studying this manual?  Even reading this paragraph slows you down?  We might have a problem, as you will have to do some guess work, and might not become very proficient unless you have a very solid intuition....\par
\par
\pard{}Let me use here, as a quick tutorial, an actual reply of mine to a {\f1{}recode} user, who writes:\par
\par
{\pard\li720{}My situation is this--I occasionally get email with special characters in it.  Sometimes this mail is from a user using IBM software and sometimes it is a user using Mac software.  I myself am on a SPARC Solaris machine.\par
\pard{}}\par
\pard{}Your situation is similar to mine, except that I {\i often} receive email needing recoding, that is, much more than {\i occasionally}!  The usual recodings I do are Mac to Latin-1, IBM page codes to Latin-1, Easy-French to Latin-1, remove Quoted-Printable, remove Base64.  These are so frequent that I made myself a few two-keystroke Emacs commands to filter the Emacs region.  This is very convenient for me.  I also resort to many other email conversions, yet more rarely than the frequent cases above.\par
\par
{\pard\li720{}It {\i seems} like this should be doable using {\f1{}recode}.  However, when I try something like {\f1{}grecode mac macfile.txt} I get nothing out--no error, no output, nothing.\par
\pard{}}\par
\pard{}Presuming you are using some recent version of {\f1{}recode}, the command:\par
\par
{\pard\keep\li720\f1{}recode mac macfile.txt\par
\pard\f0{}}\par
\pard{}is a request for recoding {\f1{}macfile.txt} over itself, overwriting the original, from Macintosh usual character code and Macintosh end of lines, to Latin-1 and Unix end of lines.  This is overwrite mode.  If you want to use {\f1{}recode} as a filter, which is probably what you need, rather do:\par
\par
{\pard\keep\li720\f1{}recode mac\par
\pard\f0{}}\par
\pard{}and give your Macintosh file as standard input, you'll get the Latin-1 file on standard output.  The above command is an abbreviation for any of:\par
\par
{\pard\keep\li720\f1{}recode mac..\line
recode mac..l1\line
recode mac..Latin-1\line
recode mac/CR..Latin-1/\line
recode Macintosh..ISO_8859-1\line
recode Macintosh/CR..ISO_8859-1/\par
\pard\f0{}}\par
\pard{}That is, a {\f1{}CR} surface, encoding newlines with ASCII <CR>, is first to be removed (this is a default surface for {\f1{}mac}), then the Macintosh charset is converted to Latin-1 and no surface is added to the result (there is no default surface for {\f1{}l1}).  If you want {\f1{}mac} code converted, but you know that newlines are already coded the Unix way, just do:\par
\par
{\pard\keep\li720\f1{}recode mac/\par
\pard\f0{}}\par
\pard{}the slash then overriding the default surface with empty, that is, none.  Here are other easy recipes:\par
\par
{\pard\keep\li720\f1{}recode pc          to filter IBM-PC code and CR-LF (default) to Latin-1\line
recode pc/         to filter IBM-PC code to Latin-1\line
recode 850         to filter code page 850 and CR-LF (default) to Latin-1\line
recode 850/        to filter code page 850 to Latin-1\line
recode /qp         to remove quoted printable\par
\pard\f0{}}\par
\pard{}The last one is indeed equivalent to any of:\par
\par
{\pard\keep\li720\f1{}recode /qp..\line
recode l1/qp..l1/\line
recode ISO_8859-1/Quoted-Printable..ISO_8859-1/\par
\pard\f0{}}\par
\pard{}Here are some reverse recipes:\par
\par
{\pard\keep\li720\f1{}recode ..mac       to filter Latin-1 to Macintosh code and CR (default)\line
recode ..mac/      to filter Latin-1 to Macintosh code\line
recode ..pc        to filter Latin-1 to IBM-PC code and CR-LF (default)\line
recode ..pc/       to filter Latin-1 to IBM-PC code\line
recode ..850       to filter Latin-1 to code page 850 and CR-LF (default)\line
recode ..850/      to filter Latin-1 to code page 850\line
recode ../qp       to force quoted printable\par
\pard\f0{}}\par
\pard{}In all the above calls, replace {\f1{}recode} by {\f1{}recode -f} if you want to proceed despite recoding errors.  If you do not use {\f1{}-f} and there is an error, the recoding output will be interrupted after first error in filter mode, or the file will not be replaced by a recoded copy in overwrite mode.\par
\par
\pard{}You may use {\f1{}recode -l} to get a list of available charsets and surfaces, and {\f1{}recode --help} to get a quick summary of options.  The above output is meant for those having already read this manual, so let me dare a suggestion: why could not you find a few more minutes in your schedule to peek further down, right into the following chapters!\par
\par
\page\pard Node: {\b Introduction}, \keepn Next: {\uldb Invoking recode}{\v Invoking_recode}, Prev: {\uldb Tutorial}{\v Tutorial}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Introduction}
#{\footnote Introduction}
${\footnote \pard{}Terminology and purpose}\par
\pard{\fs24\b Terminology and purpose}\par
\par
\pard{}A few terms are used over and over in this manual, our wise reader will learn their meaning right away.  Both ISO (International Organization for Standardisation) and IETF (Internet Engineering Task Force) have their own terminology, this document does not try to stick to either one in a strict way, while it does not want to throw more confusion in the field.  On the other hand, it would not be efficient using paraphrases all the time, so {\f1{}recode} coins a few short words, which are explained below.\par
\par
\pard{}A \'A2charset\'A2, in the context of {\f1{}recode}, is a particular association between computer codes on one side, and a repertoire of intended characters on the other side.  Codes are usually taken from a set of consecutive small integers, starting at 0.  Some characters have a graphical appearance (glyph) or displayable effect, others have special uses like, for example, to control devices or to interact with neighbouring codes to specify them more precisely.  So, a {\i charset} is roughly one of those tables, giving a meaning to each of the codes from the set of allowable values.  MIME also uses the term charset with approximately the same meaning.  It does {\i not} exactly corresponds to what ISO calls a \'A2coded character set\'A2, that is, a set of characters with an encoding for them.  An coded character set does not necessarily use all available code positions, while a MIME charset usually tries to specify them all.  A MIME charset might be the union of a few disjoint coded character sets.\par
\par
\pard{}A \'A2surface\'A2 is a term used in {\f1{}recode} only, and is a short for surface transformation of a charset stream.  This is any kind of mapping, usually reversible, which associates physical bits in some medium for a stream of characters taken from one or more charsets (usually one).  A surface is a kind of varnish added over a charset so it fits in actual bits and bytes.  How end of lines are exactly encoded is not really pertinent to the charset, and so, there is surface for end of lines.  {\f1{}Base64} is also a surface, as we may encode any charset in it.  Other examples would {\f1{}DES} enciphering, or {\f1{}gzip} compression (even if {\f1{}recode} does not offer them currently): these are ways to give a real life to theoretical charsets.  The \'A2trivial\'A2 surface consists into putting characters into fixed width little chunks of bits, usually eight such bits per character.  But things are not always that simple.\par
\par
\pard{}This {\f1{}recode} library, and the program by that name, have the purpose of converting files between various charsets and surfaces.  When this cannot be done in exact ways, as it is often the case, the program may get rid of the offending characters or fall back on approximations.  This library recognises or produces around 175 such charsets under 500 names, and handle a dozen surfaces.  Since it can convert each charset to almost any other one, many thousands of different conversions are possible.\par
\par
\pard{}The {\f1{}recode} program and library do not usually know how to split and sort out textual and non-textual information which may be mixed in a single input file.  For example, there is no surface which currently addresses the problem of how lines are blocked into physical records, when the blocking information is added as binary markers or counters within files.  So, {\f1{}recode} should be given textual streams which are rather {\i pure}.\par
\par
\pard{}This tool pays special attention to superimposition of diacritics for some French representations.  This orientation is mostly historical, it does not impair the usefulness, generality or extensibility of the program.\par
\par
\pard{}The program {\f1{}recode} has been written by Franc,ois Pinard.  With time, it got to reuse works from other contributors, and notably, those of Keld Simonsen.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Charset overview}{\v Charset_overview}\tab Overview of charsets\par
{\uldb Surface overview}{\v Surface_overview}\tab Overview of surfaces\par
{\uldb Contributing}{\v Contributing}\tab Contributions and bug reports\par
\pard{}}\par
\page\pard Node: {\b Charset overview}, \keepn Next: {\uldb Surface overview}{\v Surface_overview}, Prev: {\uldb Introduction}{\v Introduction}, Up: {\uldb Introduction}{\v Introduction}\line
K{\footnote K Charset overview}
#{\footnote Charset_overview}
${\footnote \pard{}Overview of charsets}\par
\pard{\fs24\b Overview of charsets}\par
\par
\pard{}Recoding is currently possible between many charsets, the bulk of which is described by RFC 1345 tables.  See {\uldb Tabular}{\v Tabular}.  The {\f1{}recode} library also handles some charsets in some specialised ways.  These are:\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}6-bit charsets based on CDC display code: 6/12 code from NOS; bang-bang code from Universite' de Montre'al;\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}7-bit ASCII: without any diacritics, or else: using backspace for overstriking; Unisys' Icon convention; TeX/LaTeX coding; easy French conventions for electronic mail;\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}8-bit extensions to ASCII: ISO Latin-1, Atari ST code, IBM's code for the PC, Apple's code for the Macintosh;\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}8-bit non-ASCII codes: three flavours of EBCDIC;\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}16-bit or 31-bit universal characters, and their transfer encodings.\par
\par
\pard{}}\par
\pard{}The introduction of RFC 1345 in {\f1{}recode} has brought with it a few charsets having the functionality of older ones, but yet being different in subtle ways.  The effects have not been fully investigated yet, so for now, clashes are avoided, the old and new charsets are kept well separate.\par
\par
\page\pard Node: {\b Surface overview}, \keepn Next: {\uldb Contributing}{\v Contributing}, Prev: {\uldb Charset overview}{\v Charset_overview}, Up: {\uldb Introduction}{\v Introduction}\line
K{\footnote K Surface overview}
#{\footnote Surface_overview}
${\footnote \pard{}Overview of surfaces}\par
\pard{\fs24\b Overview of surfaces}\par
\par
\pard{}For various practical considerations, it sometimes happens that the codes making up a text, written in a particular charset, cannot simply be put out in a file one after another without creating problems or breaking other things.  Sometimes, 8-bit codes cannot be written on a 7-bit medium, variable length codes need kind of envelopes, newlines require special treatment, etc.  We sometimes have to apply \'A2surfaces\'A2 to a stream of codes, which surfaces are kind of tricks used to fit the charset into those practical constraints.  Moreover, similar surfaces or tricks may be useful for many unrelated charsets, and many surfaces can be used at once over a single charset.\par
\par
\pard{}So, {\f1{}recode} has machinery to describe a combination of a charset with surfaces used over it in a file.  We would use the expression \'A2pure charset\'A2 for referring to a charset free of any surface, that is, the conceptual association between integer codes and character intents.\par
\par
\pard{}It is not always clear if some transformation will yield a charset or a surface, especially for those transformations which are only meaningful over a single charset.  The {\f1{}recode} library is not overly picky as identifying surfaces as such: when it is practical to consider a specialised surface as if it were a charset, this is preferred, and done.\par
\par
\page\pard Node: {\b Contributing}, \keepn Next: {\uldb }{\v }, Prev: {\uldb Surface overview}{\v Surface_overview}, Up: {\uldb Introduction}{\v Introduction}\line
K{\footnote K Contributing}
#{\footnote Contributing}
${\footnote \pard{}Contributions and bug reports}\par
\pard{\fs24\b Contributions and bug reports}\par
\par
\pard{}Even being the {\f1{}recode} author and current maintainer, I am no specialist in charset standards.  I only made {\f1{}recode} along the years to solve my own needs, but felt it was applicable for the needs of others.  Some FSF people liked the program structure and suggested to make it more widely available.  I often rely on {\f1{}recode} users suggestions to decide what is best to be done next.\par
\par
\pard{}Properly protecting {\f1{}recode} about possible copyright fights is a pain for me and for contributors, but we cannot avoid addressing the issue in the long run.  Besides, the Free Software Foundation, which mandates the GNU project, is very sensible to this matter.  GNU standards suggest that we stay cautious before looking at copyrighted code.  The safest and simplest way for me is to gather ideas and reprogram them anew, even if this might slow me down considerably.  For contributions going beyond a few lines of code here and there, the FSF definitely requires employer disclaimers and copyright assignments in writing.\par
\par
\pard{}When you contribute something to {\f1{}recode}, {\i please} explain what it is about.  Do not take for granted that I know those charsets which are familiar to you.  Once again, I'm no expert, and you have to help me.  Your explanations could well find their way into this documentation, too.  Also, for contributing new charsets or new surfaces, as much as possible, please provide good, solid, verifiable references for the tables you used({\ul 1}{\v Contributing_1}).\par
\par
\pard{}Many users contributed to {\f1{}recode} already, I am grateful to them for their interest and involvement.  Some suggestions can be integrated quickly while some others have to be delayed, I have to draw a line somewhere when time comes to make a new release, about what would go in it and what would go in the next.\par
\par
\pard{}Please send suggestions, documentation errors and bug reports to <recode-bugs@iro.umontreal.ca> or, if you prefer, directly to Franc,ois Pinard({\ul 2}{\v Contributing_2}).  Do not be afraid to report details, because this program is the mere aggregation of hundreds of details.\par
\par
\page\pard#{\footnote Contributing_1}
\pard{}(1) I'm not prone at accepting a charset you just invented, and which nobody uses yet: convince your friends and community first!\par
\par
\page\pard#{\footnote Contributing_2}
\pard{}(2) Use <pinard@iro.umontreal.ca> to reach me\par
\par
\page\pard Node: {\b Invoking recode}, \keepn Next: {\uldb Library}{\v Library}, Prev: {\uldb Introduction}{\v Introduction}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Invoking recode}
#{\footnote Invoking_recode}
${\footnote \pard{}How to use this program}\par
\pard{\fs24\b How to use this program}\par
\par
\pard{}With the synopsis of the {\f1{}recode} call, we stress the difference between using this program as a file filter, or recoding many files at once.  The first parameter of any call states the recoding request, and this deserves a section on its own.  Options are then presented, but somewhat grouped according to the related functionalities they control.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Synopsis}{\v Synopsis}\tab Synopsis of {\f1{}recode} call\par
{\uldb Requests}{\v Requests}\tab The {\i request} parameter\par
{\uldb Listings}{\v Listings}\tab Asking for various lists\par
{\uldb Recoding}{\v Recoding}\tab Controlling how files are recoded\par
{\uldb Reversibility}{\v Reversibility}\tab Reversibility issues\par
{\uldb Sequencing}{\v Sequencing}\tab Selecting sequencing methods\par
{\uldb Mixed}{\v Mixed}\tab Using mixed charset input\par
{\uldb Emacs}{\v Emacs}\tab Using {\f1{}recode} within Emacs\par
\pard{}}\par
\page\pard Node: {\b Synopsis}, \keepn Next: {\uldb Requests}{\v Requests}, Prev: {\uldb Invoking recode}{\v Invoking_recode}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Synopsis}
#{\footnote Synopsis}
${\footnote \pard{}Synopsis of {\f1{}recode} call}\par
\pard{\fs24\b Synopsis of {\f1{}recode} call}\par
\par
\pard{}The general format of the program call is one of:\par
\par
{\pard\keep\li720\f1{}recode [{\i option}]... [{\i charset} | {\i request} [{\i file}]... ]\par
\pard\f0{}}\par
\pard{}Some calls are used only to obtain lists produced by {\f1{}recode} itself, without actually recoding any file.  They are recognised through the usage of listing options, and these options decide what meaning should be given to an optional {\i charset} parameter.  See {\uldb Listings}{\v Listings}.\par
\par
\pard{}In other calls, the first parameter ({\i request}) always explains which transformations are expected on the files.  There are many variations to the aspect of this parameter.  We will discuss more complex situations later (see {\uldb Requests}{\v Requests}), but for many simple cases, this parameter merely looks like({\ul 1}{\v Synopsis_1}):\par
\par
{{\i \pard\li720\f1{}before}..{\i after}\par
\pard\f0{}}\par
\pard{}where {\i before} and {\i after} each gives the name of a charset.  Each {\i file} will be read assuming it is coded with charset {\i before}, it will be recoded over itself so to use the charset {\i after}.  If there is no {\i file} on the {\f1{}recode} command, the program rather acts as a Unix filter and transforms standard input onto standard output.\par
\par
\pard{}The capability of recoding many files at once is very convenient.  For example, one could easily prepare a distribution from Latin-1 to MSDOS, this way:\par
\par
{\pard\keep\li720\f1{}mkdir package\line
cp -p Makefile *.[ch] package\line
recode Latin-1..MSDOS package/*\line
zoo ah package.zoo package/*\line
rm -rf package\par
\pard\f0{}}\par
\pard{}(In this example, the non-mandatory {\f1{}-p} option to {\f1{}cp} is for preserving timestamps, and the {\f1{}zoo} program is an archiver from Rahul Dhesi which once was quite popular.)\par
\par
\pard{}The filter operation is especially useful when the input files should not be altered.  Let us make an example to illustrate this point.  Suppose that someone has a file named {\f1{}datum.txt}, which is almost a TeX file, except that diacriticised characters are written using Latin-1.  To complete the recoding of the diacriticised characters {\i only} and produce a file {\f1{}datum.tex}, without destroying the original, one could do:\par
\par
{\pard\keep\li720\f1{}cp -p datum.txt datum.tex\line
recode -d l1..tex datum.tex\par
\pard\f0{}}\par
\pard{}However, using {\f1{}recode} as a filter will achieve the same goal more neatly:\par
\par
{\pard\keep\li720\f1{}recode -d l1..tex <datum.txt >datum.tex\par
\pard\f0{}}\par
\pard{}This example also shows that {\f1{}l1} could be used instead of {\f1{}Latin-1}; charset names often have such aliases.\par
\par
\page\pard#{\footnote Synopsis_1}
\pard{}(1) In previous versions or {\f1{}recode}, a single colon {\f1{}:} was used instead of the two dots {\f1{}..} for separating charsets, but this was creating problems because colons are allowed in official charset names.  The old request syntax is still recognised for compatibility purposes, but is deprecated.\par
\par
\page\pard Node: {\b Requests}, \keepn Next: {\uldb Listings}{\v Listings}, Prev: {\uldb Synopsis}{\v Synopsis}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Requests}
#{\footnote Requests}
${\footnote \pard{}The {\i request} parameter}\par
\pard{\fs24\b The {\i request} parameter}\par
\par
\pard{}In the case where the {\i request} is merely written as {\i before}..{\i after}, then {\i before} and {\i after} specify the start charset and the goal charset for the recoding.\par
\par
\pard{}For {\f1{}recode}, charset names may contain any character, besides a comma, a forward slash, or two periods in a row.  But in practice, charset names are currently limited to alphabetic letters (upper or lower case), digits, hyphens, underlines, periods, colons or round parentheses.\par
\par
\pard{}The complete syntax for allowable {\i request} allows for unusual things, which might surprise at first.  (Do not pay too much attention to these facilities on a first reading.)  For example, {\i request} may also contain intermediary charsets, like in:\par
\par
{{\i \pard\li720\f1{}before}..{\i interim1}..{\i interim2}..{\i after}\par
\pard\f0{}}\par
\pard{}meaning that {\f1{}recode} should internally produce the {\i interim1} charset from the start charset, then work out of this {\i interim1} charset to internally produce {\i interim2}, and from there towards the goal charset.  In fact, {\f1{}recode} internally combines recipes and automatically uses interim charsets, when there is no direct recipe for transforming {\i before} into {\i after}.  But there might be many ways to do it.  When many routes are possible, the above \'A2chaining\'A2 syntax may be used to more precisely force the program towards a particular route, which it might not have naturally selected otherwise.  On the other hand, because {\f1{}recode} tries to choose good routes, chaining is only needed to achieve some rare, unusual effects.\par
\par
\pard{}Moreover, many such requests (sub-requests, more precisely) may be separated with commas (but no spaces at all), indicating a sequence of recodings, where the output of one has to serve as the input of the following one.  For example, the two following requests are equivalent:\par
\par
{{\i \pard\li720\f1{}before}..{\i interim1}..{\i interim2}..{\i after}\line
{\i before}..{\i interim1},{\i interim1}..{\i interim2},{\i interim2}..{\i after}\par
\pard\f0{}}\par
\pard{}In this example, the charset input for any recoding sub-request is identical to the charset output by the preceding sub-request.  But it does not have to be in the general case.  One might wonder what would be the meaning of declaring the charset input for a recoding sub-request of being of different nature than the charset output by a preceding sub-request, when recodings are chained in this way.  These strange usages might have a meaning and be useful for the {\f1{}recode} expert, but they are quite uncommon in practice.\par
\par
\pard{}More useful is the distinction between the concept of charset, and the concept of surfaces.  An encoded charset is represented by:\par
\par
{{\i \pard\li720\f1{}pure-charset}/{\i surface1}/{\i surface2}...\par
\pard\f0{}}\par
\pard{}using slashes to introduce surfaces, if any.  The order of application of surfaces is usually important, they cannot be freely commuted.  In the given example, {\i surface1} is first applied over the {\i pure-charset}, then {\i surface2} is applied over the result.  Given this request:\par
\par
{{\i \pard\li720\f1{}before}/{\i surface1}/{\i surface2}..{\i after}/{\i surface3}\par
\pard\f0{}}\par
\pard{}the {\f1{}recode} program will understand that the input files should have {\i surface2} removed first (because it was applied last), then {\i surface1} should be removed.  The next step will be to translate the codes from charset {\i before} to charset {\i after}, prior to applying {\i surface3} over the result.\par
\par
\pard{}Some charsets have one or more {\i implied} surfaces.  In this case, the implied surfaces are automatically handled merely by naming the charset, without any explicit surface to qualify it.  Let's take an example to illustrate this feature.  The request {\f1{}pc..l1} will indeed decode MS-DOS end of lines prior to converting IBM-PC codes to Latin-1, because {\f1{}pc} is the name of a charset({\ul 1}{\v Requests_1}) which has {\f1{}CR-LF} for its usual surface.  The request {\f1{}pc/..l1} will {\i not} decode end of lines, since the slash introduces surfaces, and even if the surface list is empty, it effectively defeats the automatic removal of surfaces for this charset.  So, empty surfaces are useful, indeed!\par
\par
\pard{}Both charsets and surfaces may have predefined alternate names, or aliases.  However, and this is rather important to understand, implied surfaces are attached to individual aliases rather than on genuine charsets.  Consequently, the official charset name and all of its aliases do not necessarily share the same implied surfaces.  The charset and all its aliases may each have its own different set of implied surfaces.\par
\par
\pard{}Charset names, surface names, or their aliases may always be abbreviated to any unambiguous prefix.  Internally in {\f1{}recode}, disambiguating tables are kept separate for charset names and surface names.\par
\par
\pard{}While recognising a charset name or a surface name (or aliases thereof), {\f1{}recode} ignores all characters besides letters and digits, so for example, the hyphens and underlines being part of an official charset name may safely be omitted (no need to un-confuse them!).  There is also no distinction between upper and lower case for charset or surface names.\par
\par
\pard{}One of the {\i before} or {\i after} keywords may be omitted.  If the double dot separator is omitted too, then the charset is interpreted as the {\i before} charset.({\ul 2}{\v Requests_2})\par
\par
\pard{}When a charset name is omitted or left empty, the value of the {\f1{}DEFAULT_CHARSET} variable in the environment is used instead.  If this variable is not defined, the {\f1{}recode} library uses an {\i usual charset} which is installation dependent.  When this variable is defined but has an empty value, charset names may just not be omitted.  Unless overriden when {\f1{}recode} was installed, this usual charset is {\f1{}Latin-1} for Unix systems or {\f1{}IBM-PC} for MS-DOS machines.\par
\par
\pard{}If the charset name is omitted but followed by surfaces, the surfaces then qualify the usual or default charset.  For example, the request {\f1{}../x} is sufficient for applying an hexadecimal surface to the input text({\ul 3}{\v Requests_3}).\par
\par
\pard{}The allowable values for {\i before} or {\i after} charsets, and various surfaces, are described in the remainder of this document.\par
\par
\page\pard#{\footnote Requests_1}
\pard{}(1) More precisely, {\f1{}pc} is an alias for the charset {\f1{}IBM-PC}.\par
\par
\page\pard#{\footnote Requests_2}
\pard{}(2) Both {\i before} and {\i after} may be omitted, in which case the double dot separator may not.  This would not be very meaningful, as the recoding reduces to a mere copy in that case.\par
\par
\page\pard#{\footnote Requests_3}
\pard{}(3) MS-DOS is one of those systems for which the default charset has implied surfaces, {\f1{}CR-LF} here.  Such surfaces are automatically removed or applied whenever the default charset is read or written, exactly as it would go for any other charset.  In the example above, on such systems, the hexadecimal surface would then {\i replace} the implied surfaces.  For {\i adding} an hexadecimal surface without removing any, one should write the request as {\f1{}/../x}.\par
\par
\page\pard Node: {\b Listings}, \keepn Next: {\uldb Recoding}{\v Recoding}, Prev: {\uldb Requests}{\v Requests}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Listings}
#{\footnote Listings}
${\footnote \pard{}Asking for various lists}\par
\pard{\fs24\b Asking for various lists}\par
\par
\pard{}Many options control listing output generated by {\f1{}recode} itself, they are not meant to accompany actual file recodings.  These options are:\par
\par
{\pard{}{\f1{}--version}\par
\pard\li720{}The program merely prints its version numbers on standard output, and exits without doing anything else.\par
\par
\pard{}{\f1{}--help}\par
\pard\li720{}The program merely prints a page of help on standard output, and exits without doing any recoding.\par
\par
\pard{}{\f1{}-C}\par
\pard{}{\f1{}--copyright}\par
\pard\li720{}Given this option, all other parameters and options are ignored.  The program prints briefly the copyright and copying conditions.  See the file {\f1{}COPYING} in the distribution for full statement of the Copyright and copying conditions.\par
\par
\pard{}{\f1{}-h[{\i language}/][{\i name}]}\par
\pard{}{\f1{}--header[=[{\i language}/][{\i name}]]}\par
\pard\li720{}Instead of recoding files, {\f1{}recode} writes a {\i language} source file on standard output and exits.  This source is meant to be included in a regular program written in the same programming {\i language}: its purpose is to declare and initialise an array, named {\i name}, which represents the requested recoding.  The only acceptable values for {\i language} are {\f1{}c} or {\f1{}perl}, and may may be abbreviated.  If {\i language} is not specified, {\f1{}c} is assumed.  If {\i name} is not specified, then it defaults to {\f1{}{\i before}_{\i after}}.  Strings {\i before} and {\i after} are cleaned before being used according to the syntax of {\i language}.\par
\par
\pard\li720{}Even if {\f1{}recode} tries its best, this option does not always succeed in producing the requested source table.  It will however, provided the recoding can be internally represented by only one step after the optimisation phase, and if this merged step conveys a one-to-one or a one-to-many explicit table.  Also, when attempting to produce sources tables, {\f1{}recode} relaxes its checking a tiny bit: it ignores the algorithmic part of some tabular recodings, it also avoids the processing of implied surfaces.  But this is all fairly technical.  Better try and see!\par
\par
\pard\li720{}Beware that other options might affect the produced source tables, these are: {\f1{}-d}, {\f1{}-g} and, particularly, {\f1{}-s}.\par
\par
\pard{}{\f1{}-k {\i pairs}}\par
\pard{}{\f1{}--known={\i pairs}}\par
\pard\li720{}This particular option is meant to help identifying an unknown charset, using as hints some already identified characters of the charset.  Some examples will help introducing the idea.\par
\par
\pard\li720{}Let's presume here that {\f1{}recode} is run on Unix, where Latin-1 is the usual charset, and that {\f1{}DEFAULT_CHARSET} is unset in the environment.  Suppose you have guessed that code 130 (decimal) of the unknown charset represents a lower case {\f1{}e} with an acute accent.  That is to say that this code should map to code 233 (decimal) in the usual charset.  By executing:\par
\par
{\pard\keep\li1440\f1{}recode -k 130:233\par
\pard\li720\f0{}}\par
\pard\li720{}you should obtain a listing similar to:\par
\par
{\pard\keep\li1440\f1{}AtariST atarist\line
CWI cphu cwi cwi2\line
IBM437 437 cp437 ibm437\line
IBM850 850 cp850 ibm850\line
IBM851 851 cp851 ibm851\line
IBM852 852 cp852 ibm852\line
IBM857 857 cp857 ibm857\line
IBM860 860 cp860 ibm860\line
IBM861 861 cp861 cpis ibm861\line
IBM863 863 cp863 ibm863\line
IBM865 865 cp865 ibm865\par
\pard\li720\f0{}}\par
\pard\li720{}You can give more than one clue at once, to restrict the list further.  Suppose you have {\i also} guessed that code 211 of the unknown charset represents an upper case {\f1{}E} with diaeresis, that is, code 203 in the usual charset.  By requesting:\par
\par
{\pard\keep\li1440\f1{}recode -k 130:233,211:203\par
\pard\li720\f0{}}\par
\pard\li720{}you should obtain:\par
\par
{\pard\keep\li1440\f1{}IBM850 850 cp850 ibm850\line
IBM852 852 cp852 ibm852\line
IBM857 857 cp857 ibm857\par
\pard\li720\f0{}}\par
\pard\li720{}The usual charset may be overridden by specifying one non-option argument.  For example, to request the list of charsets for which code 130 maps to code 142 for the Macintosh, you may ask:\par
\par
{\pard\keep\li1440\f1{}recode -k 130:142 mac\par
\pard\li720\f0{}}\par
\pard\li720{}and get:\par
\par
{\pard\keep\li1440\f1{}AtariST atarist\line
CWI cphu cwi cwi2\line
IBM437 437 cp437 ibm437\line
IBM850 850 cp850 ibm850\line
IBM851 851 cp851 ibm851\line
IBM852 852 cp852 ibm852\line
IBM857 857 cp857 ibm857\line
IBM860 860 cp860 ibm860\line
IBM861 861 cp861 cpis ibm861\line
IBM863 863 cp863 ibm863\line
IBM865 865 cp865 ibm865\par
\pard\li720\f0{}}\par
\pard\li720{}which, of course, is identical to the result of the first example, since the code 142 for the Macintosh is a small {\f1{}e} with acute.\par
\par
\pard\li720{}More formally, option {\f1{}-k} lists all possible {\i before} charsets for the {\i after} charset given as the sole non-option argument to {\f1{}recode}, but subject to restrictions given in {\i pairs}.  If there is no non-option argument, the {\i after} charset is taken to be the default charset for this {\f1{}recode}.\par
\par
\pard\li720{}The restrictions are given as a comma separated list of pairs, each pair consisting of two numbers separated by a colon.  The numbers are taken as decimal when the initial digit is between {\f1{}1} and {\f1{}9}; {\f1{}0x} starts an hexadecimal number, or else {\f1{}0} starts an octal number.  The first number is a code in any {\i before} charset, while the second number is a code in the specified {\i after} charset.  If the first number would not be transformed into the second number by recoding from some {\i before} charset to the {\i after} charset, then this {\i before} charset is rejected.  A {\i before} charset is listed only if it is not rejected by any pair.  The program will only test those {\i before} charsets having a tabular style internal description (see {\uldb Tabular}{\v Tabular}), so should be the selected {\i after} charset.\par
\par
\pard\li720{}The produced list is in fact a subset of the list produced by the option {\f1{}-l}.  As for option {\f1{}-l}, the non-option argument is interpreted as a charset name, possibly abbreviated to any non ambiguous prefix.\par
\par
\pard{}{\f1{}-l[{\i format}]}\par
\pard{}{\f1{}--list[={\i format}]}\par
\pard\li720{}This option asks for information about all charsets, or about one particular charset.  No file will be recoded.\par
\par
\pard\li720{}If there is no non-option arguments, {\f1{}recode} ignores the {\i format} value of the option, it writes a sorted list of charset names on standard output, one per line.  When a charset name have aliases or synonyms, they follow the true charset name on its line, sorted from left to right.  Each charset or alias is followed by its implied surfaces, if any.  This list is over two hundred lines.  It is best used with {\f1{}grep -i}, as in:\par
\par
{\pard\keep\li1440\f1{}recode -l | grep -i greek\par
\pard\li720\f0{}}\par
\pard\li720{}There might be one non-option argument, in which case it is interpreted as a charset name, possibly abbreviated to any non ambiguous prefix.  This particular usage of the {\f1{}-l} option is obeyed {\i only} for charsets having a tabular style internal description (see {\uldb Tabular}{\v Tabular}).  Even if most charsets have this property, some do not, then option {\f1{}-l} cannot be used to detail these particular charsets.  For knowing if a particular charset can be listed this way, you should merely try and see if this works.  The {\i format} value of the option is a keyword from the following list.  Keywords may be abbreviated by dropping suffix letters, and even reduced to the first letter only:\par
\par
{\pard\li720{}{\f1{}decimal}\par
\pard\li1440{}This format asks for the production on standard output of a concise tabular display of the charset, in which character code values are expressed in decimal.\par
\par
\pard\li720{}{\f1{}octal}\par
\pard\li1440{}This format uses octal instead of decimal in the concise tabular display of the charset.\par
\par
\pard\li720{}{\f1{}hexadecimal}\par
\pard\li1440{}This format uses hexadecimal instead of decimal in the concise tabular display of the charset.\par
\par
\pard\li720{}{\f1{}full}\par
\pard\li1440{}This format requests an extensive display of the charset on standard output, using one line per character showing its decimal, hexadecimal, octal and {\f1{}UCS-2} code values, and also a descriptive comment which should be the 10646 name for the character.\par
\par
\pard\li1440{}The descriptive comment is given in English and ASCII, yet if the English description is not available but a French one is, then the French description is given instead, using Latin-1.  However, if the {\f1{}LANGUAGE} or {\f1{}LANG} environment variable begins with the letters {\f1{}fr}, then listing preference goes to French when both descriptions are available.\par
\par
\pard\li720{}}\par
\pard\li720{}When option {\f1{}-l} is used together with a {\i charset} argument, the {\i format} defaults to {\f1{}decimal}.\par
\par
\pard{}{\f1{}-T}\par
\pard{}{\f1{}--find-subsets}\par
\pard\li720{}This option is a maintainer tool for evaluating the redundancy of those charsets, in {\f1{}recode}, which are internally represented by an {\f1{}UCS-2} data table.  After the listing has been produced, the program exits without doing any recoding.  The output of is meant to be sorted, like this: {\f1{}recode -T | sort}.  The option triggers {\f1{}recode} into comparing all pairs of charsets, seeking those which are subsets of others.  The concept and results are better explained through a few examples.  Consider these three sample lines from {\f1{}-T} output:\par
\par
{\pard\keep\li1440\f1{}[  0] IBM891 == IBM903\line
[  1] IBM1004 < CP1252\line
[ 12] INVARIANT < CSA_Z243.4-1985-1\par
\pard\li720\f0{}}\par
\pard\li720{}The first line means that {\f1{}IBM891} and {\f1{}IBM903} are completely identical as far as {\f1{}recode} is concerned, so one is fully redundant to the other.  The second line says that {\f1{}IBM1004} is wholly contained within {\f1{}CP1252}, yet there is a single character which is in {\f1{}CP1252} without being in {\f1{}IBM1004}.  The third line says that {\f1{}INVARIANT} is wholly contained within {\f1{}CSA_Z243.4-1985-1}, but twelve characters are in {\f1{}CSA_Z243.4-1985-1} without being in {\f1{}INVARIANT}.  The whole output might most probably be reduced and made more significant through a transitivity study.\par
\par
\pard{}}\par
\page\pard Node: {\b Recoding}, \keepn Next: {\uldb Reversibility}{\v Reversibility}, Prev: {\uldb Listings}{\v Listings}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Recoding}
#{\footnote Recoding}
${\footnote \pard{}Controlling how files are recoded}\par
\pard{\fs24\b Controlling how files are recoded}\par
\par
\pard{}The following options have the purpose of giving the user some fine grain control over the recoding operation themselves.\par
\par
{\pard{}{\f1{}-c}\par
\pard{}{\f1{}--colons}\par
\pard\li720{}With {\f1{}Texte} Easy French conventions, use the column {\f1{}:} instead of the double-quote {\f1{}"} for marking diaeresis.  See {\uldb Texte}{\v Texte}.\par
\par
\pard{}{\f1{}-g}\par
\pard{}{\f1{}--graphics}\par
\pard\li720{}This option is only meaningful while getting {\i out} of the {\f1{}IBM-PC} charset.  In this charset, characters 176 to 223 are used for constructing rulers and boxes, using simple or double horizontal or vertical lines.  This option forces the automatic selection of ASCII characters for approximating these rulers and boxes, at cost of making the transformation irreversible.  Option {\f1{}-g} implies {\f1{}-f}.\par
\par
\pard{}{\f1{}-t}\par
\pard{}{\f1{}--touch}\par
\pard\li720{}The {\i touch} option is meaningful only when files are recoded over themselves.  Without it, the time-stamps associated with files are preserved, to reflect the fact that changing the code of a file does not really alter its informational contents.  When the user wants the recoded files to be time-stamped at the recoding time, this option inhibits the automatic protection of the time-stamps.\par
\par
\pard{}{\f1{}-v}\par
\pard{}{\f1{}--verbose}\par
\pard\li720{}Before doing any recoding, the program will first print on {\f1{}stderr} the list of all intermediate charsets planned for recoding, starting with the {\i before} charset and ending with the {\i after} charset.  It also prints an indication of the recoding quality, as one of the word {\f1{}reversible}, {\f1{}one to one}, {\f1{}one to many}, {\f1{}many to one} or {\f1{}many to many}.\par
\par
\pard\li720{}This information will appear once or twice.  It is shown a second time only when the optimisation and step merging phase succeeds in replacing many single steps by a new one.\par
\par
\pard\li720{}This option also has a second effect.  The program will print on {\f1{}stderr} one message per {\i file} recoded, so to keep the user informed of the progress of its command.\par
\par
\pard\li720{}An easy way to know beforehand the sequence or quality of a recoding is by using the command such as:\par
\par
{\pard\keep\li1440\f1{}recode -v {\i before}..{\i after} < /dev/null\par
\pard\li720\f0{}}\par
\pard\li720{}also using the fact that, in {\f1{}recode}, an empty input file produces an empty output file.\par
\par
\pard{}{\f1{}-x {\i charset}}\par
\pard{}{\f1{}--ignore={\i charset}}\par
\pard\li720{}This option tells the program to ignore any recoding path through the specified {\i charset}, so disabling any single step using this charset as a start or end point.  This may be used when the user wants to force {\f1{}recode} into using an alternate recoding path (yet using chained requests offers a finer control, see {\uldb Requests}{\v Requests}).\par
\par
{\i \pard\li720{}charset} may be abbreviated to any unambiguous prefix.\par
\par
\pard{}}\par
\page\pard Node: {\b Reversibility}, \keepn Next: {\uldb Sequencing}{\v Sequencing}, Prev: {\uldb Recoding}{\v Recoding}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Reversibility}
#{\footnote Reversibility}
${\footnote \pard{}Reversibility issues}\par
\pard{\fs24\b Reversibility issues}\par
\par
\pard{}The following options are somewhat related to reversibility issues:\par
\par
{\pard{}{\f1{}-f}\par
\pard{}{\f1{}--force}\par
\pard\li720{}With this option, irreversible or otherwise erroneous recodings are run to completion, and {\f1{}recode} does not exit with a non-zero status if it would be only because irreversibility matters.  See {\uldb Reversibility}{\v Reversibility}.\par
\par
\pard\li720{}Without this option, {\f1{}recode} tries to protect you against recoding a file irreversibly over itself({\ul 1}{\v Reversibility_1}).  Whenever an irreversible recoding is met, or any other recoding error, {\f1{}recode} produces a warning on standard error.  The current input file does not get replaced by its recoded version, and {\f1{}recode} then proceeds with the recoding of the next file.\par
\par
\pard\li720{}When the program is merely used as a filter, standard output will have received a partially recoded copy of standard input, up to the first error point.  After all recodings have been done or attempted, and if some recoding has been aborted, {\f1{}recode} exits with a non-zero status.\par
\par
\pard\li720{}In releases of {\f1{}recode} prior to version 3.5, this option was always selected, so it was rather meaningless.  Nevertheless, users were invited to start using {\f1{}-f} right away in scripts calling {\f1{}recode} whenever convenient, in preparation for the current behaviour.\par
\par
\pard{}{\f1{}-q}\par
\pard{}{\f1{}--quiet}\par
\pard{}{\f1{}--silent}\par
\pard\li720{}This option has the sole purpose of inhibiting warning messages about irreversible recodings, and other such diagnostics.  It has no other effect, in particular, it does {\i not} prevent recodings to be aborted or {\f1{}recode} to return a non-zero exit status when irreversible recodings are met.\par
\par
\pard\li720{}This option is set automatically for the children processes, when recode splits itself in many collaborating copies.  Doing so, the diagnostic is issued only once by the parent.  See option {\f1{}-p}.\par
\par
\pard{}{\f1{}-s}\par
\pard{}{\f1{}--strict}\par
\pard\li720{}By using this option, the user requests that {\f1{}recode} be very strict while recoding a file, merely losing in the transformation any character which is not explicitly mapped from a charset to another.  Such a loss is not reversible and so, will bring {\f1{}recode} to fail, unless the option {\f1{}-f} is also given as a kind of counter-measure.\par
\par
\pard\li720{}Using {\f1{}-s} without {\f1{}-f} might render the {\f1{}recode} program very susceptible to the slighest file abnormalities.  Despite it might be irritating to some users, such paranoia is sometimes wanted and useful.\par
\par
\pard{}}\par
\pard{}Even if {\f1{}recode} tries hard at keeping the recodings reversible, you should not develop an unconditional confidence in its ability to do so.  You {\i ought} to keep only reasonable expectations about reverse recodings.  In particular, consider:\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Most transformations are fully reversible for all inputs, but lose this property whenever {\f1{}-s} is specified.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}A few transformations are not meant to be reversible, by design.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Reversibility sometimes depends on actual file contents and cannot be ascertained beforehand, without reading the file.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Reversibility is never absolute across successive versions of this program.  Even correcting a small bug in a mapping could induce slight discrepancies later.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Reversibility is easily lost by merging.  This is best explained through an example.  If you reversibly recode a file from charset {\i A} to charset {\i B}, then you reversibly recode the result from charset {\i B} to charset {\i C}, you cannot expect to recover the original file by merely recoding from charset {\i C} directly to charset {\i A}.  You will instead have to recode from charset {\i C} back to charset {\i B}, and only then from charset {\i B} to charset {\i A}.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Faulty files create a particular problem.  Consider an example, recoding from {\f1{}IBM-PC} to {\f1{}Latin-1}.  End of lines are represented as {\f1{}\'5Cr\'5Cn} in {\f1{}IBM-PC} and as {\f1{}\'5Cn} in {\f1{}Latin-1}.  There is no way by which a faulty {\f1{}IBM-PC} file containing a {\f1{}\'5Cn} not preceded by {\f1{}\'5Cr} be translated into a {\f1{}Latin-1} file, and then back.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}There is another difficulty arising from code equivalences.  For example, in a {\f1{}LaTeX} charset file, the string {\f1{}\'5C^\'5Ci\'7B\'7D} could be recoded back and forth through another charset and become {\f1{}\'5C^\'7B\'5Ci\'7D}.  Even if the resulting file is equivalent to the original one, it is not identical.\par
\par
\pard{}}\par
\pard{}Unless option {\f1{}-s} is used, {\f1{}recode} automatically tries to fill mappings with invented correspondences, often making them fully reversible.  This filling is not made at random.  The algorithm tries to stick to the identity mapping and, when this is not possible, it prefers generating many small permutation cycles, each involving only a few codes.\par
\par
\pard{}For example, here is how {\f1{}IBM-PC} code 186 gets translated to {\f1{}control-U} in {\f1{}Latin-1}.  {\f1{}Control-U} is 21.  Code 21 is the {\f1{}IBM-PC} section sign, which is 167 in {\f1{}Latin-1}.  {\f1{}recode} cannot reciprocate 167 to 21, because 167 is the masculine ordinal indicator within {\f1{}IBM-PC}, which is 186 in {\f1{}Latin-1}.  Code 186 within {\f1{}IBM-PC} has no {\f1{}Latin-1} equivalent; by assigning it back to 21, {\f1{}recode} closes this short permutation loop.\par
\par
\pard{}As a consequence of this map filling, {\f1{}recode} may sometimes produce {\i funny} characters.  They may look annoying, they are nevertheless helpful when one changes his (her) mind and wants to revert to the prior recoding.  If you cannot stand these, use option {\f1{}-s}, which asks for a very strict recoding.\par
\par
\pard{}This map filling sometimes has a few surprising consequences, which some users wrongly interpreted as bugs.  Here are two examples.\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}1. \tab{}In some cases, {\f1{}recode} seems to copy a file without recoding it.  But in fact, it does.  Consider you requested:\par
\par
{\pard\keep\li1440\f1{}recode l1..us < File-Latin1 > File-ASCII\line
cmp File-Latin1 File-ASCII\par
\pard\li720\f0{}}\par
\pard\li720{}then {\f1{}cmp} will not report any difference.  This is quite normal.  Latin-1 gets correctly recoded to ASCII for charsets commonalities (which are the first 128 characters, in this case).  The remaining last 128 Latin-1 characters have no ASCII correspondent.  Instead of losing them, recode elects to map them to unspecified characters of ASCII, so making the recoding reversible.  The simplest way of achieving this is merely to keep those last 128 characters unchanged.  The overall effect is copying the file verbatim.\par
\par
\pard\li720{}If you feel this behaviour is too generous and if you do not wish to care about reversibility, simply use option {\f1{}-s}.  By doing so, {\f1{}recode} will strictly map only those Latin-1 characters which have an ASCII equivalent, and will merely drop those which do not.  Then, there is more chance that you will observe a difference between the input and the output file.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}2. \tab{}Recoding the wrong way could sometimes give the false impression that recoding has {\i almost} been done properly.  Consider you request:\par
\par
{\pard\keep\li1440\f1{}recode 437..l1 < File-Latin1 > Temp1\line
recode 437..l1 < Temp1 > Temp2\par
\pard\li720\f0{}}\par
\pard\li720{}so declaring wrongly {\f1{}File-Latin1} to be an IBM-PC file, and recoding to Latin1.  This is surely ill defined and not meaningful.  Yet, if you repeat this step a second time, you might notice that many (not all) characters in {\f1{}Temp2} are identical to those in {\f1{}File-Latin1}.  A few people try to discover how {\f1{}recode} works by experimenting a little at random, rather than reading and understanding the documentation; such results are surely confusing, as they provide a feel to those people that they understood something.\par
\par
\pard\li720{}Reversible codings have this property that, if applied several times in the same direction, they will eventually bring any character back to its original value.  Since {\f1{}recode} seeks small permutation cycles when creating reversible codings, besides characters unchanged by the recoding, most permutation cycles will be of length 2, and fewer of length 3, etc.  So, it is just expectable that applying the recoding twice in the same direction will recover most characters, but will fail to recover those participating in permutation cycles of length 3.  On the other end, recoding six times in the same direction would recover all characters in cycles of length 1, 2, 3 or 6.\par
\par
\pard{}}\par
\page\pard#{\footnote Reversibility_1}
\pard{}(1) There are still some cases of ambiguous output which are rather difficult to detect, and for which the protection is not active.\par
\par
\page\pard Node: {\b Sequencing}, \keepn Next: {\uldb Mixed}{\v Mixed}, Prev: {\uldb Reversibility}{\v Reversibility}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Sequencing}
#{\footnote Sequencing}
${\footnote \pard{}Selecting sequencing methods}\par
\pard{\fs24\b Selecting sequencing methods}\par
\par
\pard{}This program uses a few techniques when it is discovered that many passes are needed to comply with the {\i request}.  For example, suppose that four elementary steps were selected at recoding path optimisation time.  Then {\f1{}recode} will split itself into four different interconnected tasks, logically equivalent to:\par
\par
{{\i \pard\li720\f1{}step1} <{\i input} | {\i step2} | {\i step3} | {\i step4} >{\i output}\par
\pard\f0{}}\par
\pard{}The splitting into subtasks is often done using Unix pipes.  But the splitting may also be completely avoided, and rather simulated by using memory buffer, or intermediate files.  The various {\f1{}--sequence={\i strategy}} options gives you control over the flow methods, by replacing {\i strategy} with {\f1{}memory}, {\f1{}pipe} or {\f1{}files}.  So, these options may be used to override the default behaviour, which is also explained below.\par
\par
{\pard{}{\f1{}--sequence=memory}\par
\pard\li720{}When the recoding requires a combination of two or more elementary recoding steps, this option forces many passes over the data, using in-memory buffers to hold all intermediary results.\par
\par
\pard{}{\f1{}-i}\par
\pard{}{\f1{}--sequence=files}\par
\pard\li720{}When the recoding requires a combination of two or more elementary recoding steps, this option forces many passes over the data, using intermediate files between passes.  This is the default behaviour when files are recoded over themselves.  If this option is selected in filter mode, that is, when the program reads standard input and writes standard output, it might take longer for programs further down the pipe chain to start receiving some recoded data.\par
\par
\pard{}{\f1{}-p}\par
\pard{}{\f1{}--sequence=pipe}\par
\pard\li720{}When the recoding requires a combination of two or more elementary recoding steps, this option forces the program to fork itself into a few copies interconnected with pipes, using the {\f1{}pipe(2)} system call.  All copies of the program operate in parallel.  This is the default behaviour in filter mode.  If this option is used when files are recoded over themselves, this should also save disk space because some temporary files might not be needed, at cost of more system overhead.\par
\par
\pard\li720{}If, at installation time, the {\f1{}pipe(2)} call is said to be unavailable, selecting option {\f1{}-p} is equivalent to selecting option {\f1{}-i}.\par
\par
\pard{}}\par
\page\pard Node: {\b Mixed}, \keepn Next: {\uldb Emacs}{\v Emacs}, Prev: {\uldb Sequencing}{\v Sequencing}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Mixed}
#{\footnote Mixed}
${\footnote \pard{}Using mixed charset input}\par
\pard{\fs24\b Using mixed charset input}\par
\par
\pard{}In real life and practice, textual files are often made up of many charsets at once.  Some parts of the file encode one charset, while other parts encode another charset, and so forth.  Usually, a file does not toggle between more than two or three charsets.  The means to distinguish which charsets are encoded at various places is not always easy.  The {\f1{}recode} program is able to handle only a few simple cases of mixed input.\par
\par
\pard{}The default {\f1{}recode} behaviour is to expect pure charset files, to be recoded as other pure charset files.  However, the following options allow for a few precise kinds of mixed charset files.\par
\par
{\pard{}{\f1{}-d}\par
\pard{}{\f1{}--diacritics}\par
\pard\li720{}While converting to or from one of {\f1{}HTML} or {\f1{}LaTeX} charset, limit conversion to some subset of all characters.  For {\f1{}HTML}, limit conversion to the subset of all non-ASCII characters.  For {\f1{}LaTeX}, limit conversion to the subset of all non-English letters.  This is particularly useful, for example, when people create what would be valid {\f1{}HTML}, TeX or LaTeX files, if only they were using provided sequences for applying diacritics instead of using the diacriticised characters directly from the underlying character set.\par
\par
\pard\li720{}While converting to {\f1{}HTML} or {\f1{}LaTeX} charset, this option assumes that characters not in the said subset are properly coded or protected already, {\f1{}recode} then transmit them literally.  While converting the other way, this option prevents translating back coded or protected versions of characters not in the said subset.  See {\uldb HTML}{\v HTML}.  See {\uldb LaTeX}{\v LaTeX}.\par
\par
\pard{}{\f1{}-S[{\i language}]}\par
\pard{}{\f1{}--source[={\i language}]}\par
\pard\li720{}The bulk of the input file is expected to be written in {\f1{}ASCII}, except for parts, like comments and string constants, which are written using another charset than {\f1{}ASCII}.  When {\i language} is {\f1{}c}, the recoding will proceed only with the contents of comments or strings, while everything else will be copied without recoding.  When {\i language} is {\f1{}po}, the recoding will proceed only within translator comments (those having whitespace immediately following the initial {\f1{}#}) and with the contents of {\f1{}msgstr} strings.\par
\par
\pard\li720{}For the above things to work, the non-{\f1{}ASCII} encoding of the comment or string should be such that an {\f1{}ASCII} scan will successfully find where the comment or string ends.\par
\par
\pard\li720{}Even if {\f1{}ASCII} is the usual charset for writing programs, some compilers are able to directly read other charsets, like {\f1{}UTF-8} say.  There is currently no provision in {\f1{}recode} for reading mixed charset sources which are not based on {\f1{}ASCII}.  It is probable that the need for mixed recoding is not as pressing in such cases.\par
\par
\pard\li720{}For example, after one does:\par
\par
{\pard\keep\li1440\f1{}recode -Spo pc/..u8 < {\i input}.po > {\i output}.po\par
\pard\li720\f0{}}\par
\pard\li720{}file {\f1{}{\i output}.po} holds a copy of {\f1{}{\i input}.po} in which translator comments and the contents of {\f1{}msgstr} strings {\i only} have been recoded from the {\f1{}IBM-PC} charset to pure {\f1{}UTF-8}, without attempting conversion of end-of-lines.  Machine generated comments and original {\f1{}msgid} strings are not to be touched by this recoding.\par
\par
\pard\li720{}If {\i language} is not specified, {\f1{}c} is assumed.\par
\par
\pard{}}\par
\page\pard Node: {\b Emacs}, \keepn Next: {\uldb }{\v }, Prev: {\uldb Mixed}{\v Mixed}, Up: {\uldb Invoking recode}{\v Invoking_recode}\line
K{\footnote K Emacs}
#{\footnote Emacs}
${\footnote \pard{}Using {\f1{}recode} within Emacs}\par
\pard{\fs24\b Using {\f1{}recode} within Emacs}\par
\par
\pard{}The fact {\f1{}recode} is a filter makes it quite easy to use from within GNU Emacs.  For example, recoding the whole buffer from the IBM-PC charset to current charset (Latin-1 on Unix) is easily done with:\par
\par
{\pard\keep\li720\f1{}C-x h C-u M-| recode ibmpc RET\par
\pard\f0{}}\par
\pard{}The {\f1{}C-x h} selects the whole buffer, and {\f1{}C-u M-|} filters and replaces the current region through the given shell command.  Here is another example, binding the keys {\f1{}C-c T} to the recoding of the current region from Easy French to Latin-1 (on Unix) and the key {\f1{}C-u C-c T} from Latin-1 (on Unix) to Easy French:\par
\par
{\pard\keep\li720\f1{}(global-set-key "\'5CC-cT" 'recode-texte)\line
\line
(defun recode-texte (flag)\line
  (interactive "P")\line
  (shell-command-on-region\line
   (region-beginning) (region-end)\line
   (concat "recode " (if flag "..txte" "txte")) t)\line
  (exchange-point-and-mark))\par
\pard\f0{}}\par
\page\pard Node: {\b Library}, \keepn Next: {\uldb Universal}{\v Universal}, Prev: {\uldb Invoking recode}{\v Invoking_recode}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Library}
#{\footnote Library}
${\footnote \pard{}A recoding library}\par
\pard{\fs24\b A recoding library}\par
\par
\pard{}The program named {\f1{}recode} is just an application of its recoding library.  The recoding library is available separately for other C programs.  A good way to acquire some familiarity with the recoding library is to get acquainted with the {\f1{}recode} program itself.\par
\par
\pard{}To use the recoding library once it is installed, a C program needs to have a line:\par
\par
{\pard\keep\li720\f1{}#include <recode.h>\par
\pard\f0{}}\par
\pard{}near its beginning, and the user should have {\f1{}-lrecode} on the linking call, so modules from the recoding library are found.\par
\par
\pard{}The library is still under development.  As it stands, it contains four identifiable sets of routines: the outer level functions, the request level functions, the task level functions and the charset level functions.  There are discussed in separate sections.\par
\par
\pard{}For effectively using the recoding library in most applications, it should be rarely needed to study anything beyond the main initialisation function at outer level, and then, various functions at request level.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Outer level}{\v Outer_level}\tab Outer level functions\par
{\uldb Request level}{\v Request_level}\tab Request level functions\par
{\uldb Task level}{\v Task_level}\tab Task level functions\par
{\uldb Charset level}{\v Charset_level}\tab Charset level functions\par
{\uldb Errors}{\v Errors}\tab Handling errors\par
\pard{}}\par
\page\pard Node: {\b Outer level}, \keepn Next: {\uldb Request level}{\v Request_level}, Prev: {\uldb Library}{\v Library}, Up: {\uldb Library}{\v Library}\line
K{\footnote K Outer level}
#{\footnote Outer_level}
${\footnote \pard{}Outer level functions}\par
\pard{\fs24\b Outer level functions}\par
\par
\pard{}The outer level functions mainly prepare the whole recoding library for use, or do actions which are unrelated to specific recodings.  Here is an example of a program which does not really make anything useful.\par
\par
{{\pard\keep\li720\f1{}#include <stdbool.h>\line
#include <recode.h>\line
\line
int\line
main ()\line
\'7B\line
  RECODE_OUTER outer = recode_new_outer (true);\line
\line
  recode_delete_outer (outer);\line
  exit (0);\line
\'7D\par
\pard\keep\li720{}}}\par
\pard\f0{}The header file {\f1{}<recode.h>} declares an opaque {\f1{}RECODE_OUTER} structure, which the programmer should use for allocating a variable in his program (let's assume the programmer is a male, here, no prejudice intended).  This {\f1{}outer} variable is given as a first argument to all outer level functions.\par
\par
\pard{}The {\f1{}<recode.h>} header file uses the Boolean type setup by the system header file {\f1{}<stdbool.h>}.  But this header file is still fairly new in C standards, and likely does not exist everywhere.  If you system does not offer this system header file yet, the proper compilation of the {\f1{}<recode.h>} file could be guaranteed through the replacement of the inclusion line by:\par
\par
{\pard\keep\li720\f1{}typedef enum \'7Bfalse = 0, true = 1\'7D bool;\par
\pard\f0{}}\par
\pard{}People wanting wider portability, or Autoconf lovers, might arrange their {\f1{}configure.in} for being able to write something more general, like:\par
\par
{{\pard\keep\li720\f1{}#if STDC_HEADERS\line
# include <stdlib.h>\line
#endif\line
\line
/* Some systems do not define EXIT_*, even with STDC_HEADERS.  */\line
#ifndef EXIT_SUCCESS\line
# define EXIT_SUCCESS 0\line
#endif\line
#ifndef EXIT_FAILURE\line
# define EXIT_FAILURE 1\line
#endif\line
/* The following test is to work around the gross typo in systems like Sony\line
   NEWS-OS Release 4.0C, whereby EXIT_FAILURE is defined to 0, not 1.  */\line
#if !EXIT_FAILURE\line
# undef EXIT_FAILURE\line
# define EXIT_FAILURE 1\line
#endif\line
\line
#if HAVE_STDBOOL_H\line
# include <stdbool.h>\line
#else\line
typedef enum \'7Bfalse = 0, true = 1\'7D bool;\line
#endif\line
\line
#include <recode.h>\line
\line
int\line
main ()\line
\'7B\line
  RECODE_OUTER outer = recode_new_outer (true);\line
\line
  recode_term_outer (outer);\line
  exit (EXIT_SUCCESS);\line
\'7D\par
\pard\keep\li720{}}}\par
\pard\f0{}but we will not insist on such details in the examples to come.\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Initialisation functions\par
\par
{\pard\keep\li1440\f1{}RECODE_OUTER recode_new_outer ({\i auto_abort});\line
bool recode_delete_outer ({\i outer});\par
\pard\li720\f0{}}\par
\pard\li720{}The recoding library absolutely needs to be initialised before being used, and {\f1{}recode_new_outer} has to be called once, first.  Besides the {\i outer} it is meant to initialise, the function accepts a Boolean argument whether or not the library should automatically issue diagnostics on standard and abort the whole program on errors.  When {\i auto_abort} is {\f1{}true}, the library later conveniently issues diagnostics itself, and aborts the calling program on errors.  This is merely a convenience, because if this parameter was {\f1{}false}, the calling program should always take care of checking the return value of all other calls to the recoding library functions, and when any error is detected, issue a diagnostic and abort processing itself.\par
\par
\pard\li720{}Regardless of the setting of {\i auto_abort}, all recoding library functions return a success status.  Most functions are geared for returning {\f1{}false} for an error, and {\f1{}true} if everything went fine.  Functions returning structures or strings return {\f1{}NULL} instead of the result, when the result cannot be produced.  If {\i auto_abort} is selected, functions either return {\f1{}true}, or do not return at all.\par
\par
\pard\li720{}As in the example above, {\f1{}recode_new_outer} is called only once in most cases.  Calling {\f1{}recode_new_outer} implies some overhead, so calling it more than once should preferably be avoided.\par
\par
\pard\li720{}The termination function {\f1{}recode_delete_outer} reclaims the memory allocated by {\f1{}recode_new_outer} for a given {\i outer} variable.  Calling {\f1{}recode_delete_outer} prior to program termination is more aesthetic then useful, as all memory resources are automatically reclaimed when the program ends.  You may spare this terminating call if you prefer.\par
\par
\pard{}}\par
\page\pard Node: {\b Request level}, \keepn Next: {\uldb Task level}{\v Task_level}, Prev: {\uldb Outer level}{\v Outer_level}, Up: {\uldb Library}{\v Library}\line
K{\footnote K Request level}
#{\footnote Request_level}
${\footnote \pard{}Request level functions}\par
\pard{\fs24\b Request level functions}\par
\par
\pard{}The request level functions are meant to cover most recoding needs programmers may have; they should provide all usual functionality.  Their API is almost stable by now.  To get started with request level functions, here is a full example of a program which sole job is to filter {\f1{}ibmpc} code on its standard input into {\f1{}latin1} code on its standard output.\par
\par
{{\pard\keep\li720\f1{}#include <stdio.h>\line
#include <stdbool.h>\line
#include <recode.h>\line
\line
int\line
main ()\line
\'7B\line
  RECODE_OUTER outer = recode_new_outer (true);\line
  RECODE_REQUEST request = recode_new_request (outer);\line
  bool success;\line
\line
  recode_scan_request (request, "ibmpc..latin1");\line
\line
  success = recode_file_to_file (request, stdin, stdout);\line
\line
  recode_delete_request (request);\line
  recode_delete_outer (outer);\line
\line
  exit (success ? 0 : 1);\line
\'7D\par
\pard\keep\li720{}}}\par
\pard\f0{}The header file {\f1{}<recode.h>} declares a {\f1{}RECODE_REQUEST} structure, which the programmer should use for allocating a variable in his program.  This {\i request} variable is given as a first argument to all request level functions, and in most cases, may be considered as opaque.\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Initialisation functions\par
\par
{\pard\keep\li1440\f1{}RECODE_REQUEST recode_new_request ({\i outer});\line
bool recode_delete_request ({\i request});\par
\pard\li720\f0{}}\par
\pard\li720{}No {\i request} variable may not be used in other request level functions of the recoding library before having been initialised by {\f1{}recode_new_request}.  There may be many such {\i request} variables, in which case, they are independent of one another and they all need to be initialised separately.  To avoid memory leaks, a {\i request} variable should not be initialised a second time without calling {\f1{}recode_delete_request} to "un-initialise" it.\par
\par
\pard\li720{}Like for {\f1{}recode_delete_outer}, calling {\f1{}recode_delete_request} prior to program termination, in the example above, may be left out.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Fields of {\f1{}struct recode_request}\par
\par
\pard\li720{}Here are the fields of a {\f1{}struct recode_request} which may be meaningfully changed, once a {\i request} has been initialised by {\f1{}recode_new_request}, but before it gets used.  It is not very frequent, in practice, that these fields need to be changed.  To access the fields, you need to include {\f1{}recodext.h} {\i instead} of {\f1{}recode.h}, in which case there also is a greater chance that you need to recompile your programs if a new version of the recoding library gets installed.\par
\par
{\pard\li720{}{\f1{}verbose_flag}\par
\pard\li1440{}This field is initially {\f1{}false}.  When set to {\f1{}true}, the library will echo to stderr the sequence of elementary recoding steps needed to achieve the requested recoding.\par
\par
\pard\li720{}{\f1{}diaeresis_char}\par
\pard\li1440{}This field is initially the ASCII value of a double quote {\f1{}"}, but it may also be the ASCII value of a colon {\f1{}:}.  In {\f1{}texte} charset, some countries use double quotes to mark diaeresis, while other countries prefer colons.  This field contains the diaeresis character for the {\f1{}texte} charset.\par
\par
\pard\li720{}{\f1{}make_header_flag}\par
\pard\li1440{}This field is initially {\f1{}false}.  When set to {\f1{}true}, it indicates that the program is merely trying to produce a recoding table in source form rather than completing any actual recoding.  In such a case, the optimisation of step sequence can be attempted much more aggressively.  If the step sequence cannot be reduced to a single step, table production will fail.\par
\par
\pard\li720{}{\f1{}diacritics_only}\par
\pard\li1440{}This field is initially {\f1{}false}.  For {\f1{}HTML} and {\f1{}LaTeX} charset, it is often convenient to recode the diacriticized characters only, while just not recoding other HTML code using ampersands or angular brackets, or LaTeX code using backslashes.  Set the field to {\f1{}true} for getting this behaviour.  In the other charset, one can edit text as well as HTML or LaTeX directives.\par
\par
\pard\li720{}{\f1{}ascii_graphics}\par
\pard\li1440{}This field is initially {\f1{}false}, and relate to characters 176 to 223 in the {\f1{}ibmpc} charset, which are use to draw boxes.  When set to {\f1{}true}, while getting out of {\f1{}ibmpc}, ASCII characters are selected so to graphically approximate these boxes.\par
\par
\pard\li720{}}\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Study of request strings\par
\par
{\pard\keep\li1440\f1{}bool recode_scan_request ({\i request}, "{\i string}");\par
\pard\li720\f0{}}\par
\pard\li720{}The main role of a {\i request} variable is to describe a set of recoding transformations.  Function {\f1{}recode_scan_request} studies the given {\i string}, and stores an internal representation of it into {\i request}.  Note that {\i string} may be a full-fledge {\f1{}recode} request, possibly including surfaces specifications, intermediary charsets, sequences, aliases or abbreviations (see {\uldb Requests}{\v Requests}).\par
\par
\pard\li720{}The internal representation automatically receives some pre-conditioning and optimisation, so the {\i request} may then later be used many times to achieve many actual recodings.  It would not be efficient calling {\f1{}recode_scan_request} many times with the same {\i string}, it is better having many {\i request} variables instead.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Actual recoding jobs\par
\par
\pard\li720{}Once the {\i request} variable holds the description of a recoding transformation, a few functions use it for achieving an actual recoding.  Either input or output of a recoding may be string, an in-memory buffer, or a file.\par
\par
\pard\li720{}Functions with names like {\f1{}recode_{\i input-type}_to_{\i output-type}} request an actual recoding, and are described below.  It is easy to remember which arguments each function accepts, once grasped some simple principles for each possible {\i type}.  However, one of the recoding function escapes these principles and is discussed separately, first.\par
\par
{\pard\keep\li1440\f1{}recode_string ({\i request}, {\i string});\par
\pard\li720\f0{}}\par
\pard\li720{}The function {\f1{}recode_string} recodes {\i string} according to {\i request}, and directly returns the resulting recoded string freshly allocated, or {\f1{}NULL} if the recoding could not succeed for some reason.  When this function is used, it is the responsibility of the programmer to ensure that the memory used by the returned string is later reclaimed.\par
\par
{\pard\keep\li1440\f1{}char *recode_string_to_buffer ({\i request},\line
  {\i input_string},\line
  &{\i output_buffer}, &{\i output_length}, &{\i output_allocated});\line
bool recode_string_to_file ({\i request},\line
  {\i input_file},\line
  {\i output_file});\line
bool recode_buffer_to_buffer ({\i request},\line
  {\i input_buffer}, {\i input_length},\line
  &{\i output_buffer}, &{\i output_length}, &{\i output_allocated});\line
bool recode_buffer_to_file ({\i request},\line
  {\i input_buffer}, {\i input_length},\line
  {\i output_file});\line
bool recode_file_to_buffer ({\i request},\line
  {\i input_file},\line
  &{\i output_buffer}, &{\i output_length}, &{\i output_allocated});\line
bool recode_file_to_file ({\i request},\line
  {\i input_file},\line
  {\i output_file});\par
\pard\li720\f0{}}\par
\pard\li720{}All these functions return a {\f1{}bool} result, {\f1{}false} meaning that the recoding was not successful, often because of reversibility issues.  The name of the function well indicates on which types it reads and which type it produces.  Let's discuss these three types in turn.\par
\par
{\pard\li720{}string\par
\pard\li1440{}A string is merely an in-memory buffer which is terminated by a {\f1{}NUL} character (using as many bytes as needed), instead of being described by a byte length.  For input, a pointer to the buffer is given through one argument.\par
\par
\pard\li1440{}It is notable that there is no {\f1{}to_string} functions.  Only one function recodes into a string, and it is {\f1{}recode_string}, which has already been discussed separately, above.\par
\par
\pard\li720{}buffer\par
\pard\li1440{}A buffer is a sequence of bytes held in computer memory.  For input, two arguments provide a pointer to the start of the buffer and its byte size.  Note that for charsets using many bytes per character, the size is given in bytes, not in characters.\par
\par
\pard\li1440{}For output, three arguments provide the address of three variables, which will receive the buffer pointer, the used buffer size in bytes, and the allocated buffer size in bytes.  If at the time of the call, the buffer pointer is {\f1{}NULL}, then the allocated buffer size should also be zero, and the buffer will be allocated afresh by the recoding functions.  However, if the buffer pointer is not {\f1{}NULL}, it should be already allocated, the allocated buffer size then gives its size.  If the allocated size gets exceeded while the recoding goes, the buffer will be automatically reallocated bigger, probably elsewhere, and the allocated buffer size will be adjusted accordingly.\par
\par
\pard\li1440{}The second variable, giving the in-memory buffer size, will receive the exact byte size which was needed for the recoding.  A {\f1{}NUL} character is guaranteed at the end of the produced buffer, but is not counted in the byte size of the recoding.  Beyond that {\f1{}NUL}, there might be some extra space after the recoded data, extending to the allocated buffer size.\par
\par
\pard\li720{}file\par
\pard\li1440{}A file is a sequence of bytes held outside computer memory, but buffered through it.  For input, one argument provides a pointer to a file already opened for read.  The file is then read and recoded from its current position until the end of the file, effectively swallowing it in memory if the destination of the recoding is a buffer.  For reading a file filtered through the recoding library, but only a little bit at a time, one should rather use {\f1{}recode_filter_open} and {\f1{}recode_filter_close} (these two functions are not yet available).\par
\par
\pard\li1440{}For output, one argument provides a pointer to a file already opened for write.  The result of the recoding is written to that file starting at its current position.\par
\par
\pard\li720{}}\par
\pard{}}\par
\pard{}The following special function is still subject to change:\par
\par
{\pard\keep\li720\f1{}void recode_format_table ({\i request}, {\i language}, "{\i name}");\par
\pard\f0{}}\par
\pard{}and is not documented anymore for now.\par
\par
\page\pard Node: {\b Task level}, \keepn Next: {\uldb Charset level}{\v Charset_level}, Prev: {\uldb Request level}{\v Request_level}, Up: {\uldb Library}{\v Library}\line
K{\footnote K Task level}
#{\footnote Task_level}
${\footnote \pard{}Task level functions}\par
\pard{\fs24\b Task level functions}\par
\par
\pard{}The task level functions are used internally by the request level functions, they allow more explicit control over files and memory buffers holding input and output to recoding processes.  The interface specification of task level functions is still subject to change a bit.\par
\par
\pard{}To get started with task level functions, here is a full example of a program which sole job is to filter {\f1{}ibmpc} code on its standard input into {\f1{}latin1} code on its standard output.  That is, this program has the same goal as the one from the previous section, but does its things a bit differently.\par
\par
{{\pard\keep\li720\f1{}#include <stdio.h>\line
#include <stdbool.h>\line
#include <recodext.h>\line
\line
int\line
main ()\line
\'7B\line
  RECODE_OUTER outer = recode_new_outer (false);\line
  RECODE_REQUEST request = recode_new_request (outer);\line
  RECODE_TASK task;\line
  bool success;\line
\line
  recode_scan_request (request, "ibmpc..latin1");\line
\line
  task = recode_new_task (request);\line
  task->input.file = "";\line
  task->output.file = "";\line
  success = recode_perform_task (task);\line
\line
  recode_delete_task (task);\line
  recode_delete_request (request);\line
  recode_delete_outer (outer);\line
\line
  exit (success ? 0 : 1);\line
\'7D\par
\pard\keep\li720{}}}\par
\pard\f0{}The header file {\f1{}<recode.h>} declares a {\f1{}RECODE_TASK} structure, which the programmer should use for allocating a variable in his program.  This {\f1{}task} variable is given as a first argument to all task level functions.  The programmer ought to change and possibly consult a few fields in this structure, using special functions.\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Initialisation functions\par
\par
{\pard\keep\li1440\f1{}RECODE_TASK recode_new_task ({\i request});\line
bool recode_delete_task ({\i task});\par
\pard\li720\f0{}}\par
\pard\li720{}No {\i task} variable may be used in other task level functions of the recoding library without having first been initialised with {\f1{}recode_new_task}.  There may be many such {\i task} variables, in which case, they are independent of one another and they all need to be initialised separately.  To avoid memory leaks, a {\i task} variable should not be initialised a second time without calling {\f1{}recode_delete_task} to "un-initialise" it.  This function also accepts a {\i request} argument and associates the request to the task.  In fact, a task is essentially a set of recoding transformations with the specification for its current input and its current output.\par
\par
\pard\li720{}The {\i request} variable may be scanned before or after the call to {\f1{}recode_new_task}, it does not matter so far.  Immediately after initialisation, before further changes, the {\i task} variable associates {\i request} empty in-memory buffers for both input and output.  The output buffer will later get allocated automatically on the fly, as needed, by various task processors.\par
\par
\pard\li720{}Even if a call to {\f1{}recode_delete_task} is not strictly mandatory before ending the program, it is cleaner to always include it.  Moreover, in some future version of the recoding library, it might become required.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Fields of {\f1{}struct task_request}\par
\par
\pard\li720{}Here are the fields of a {\f1{}struct task_request} which may be meaningfully changed, once a {\i task} has been initialised by {\f1{}recode_new_task}.  In fact, fields are expected to change.  Once again, to access the fields, you need to include {\f1{}recodext.h} {\i instead} of {\f1{}recode.h}, in which case there also is a greater chance that you need to recompile your programs if a new version of the recoding library gets installed.\par
\par
{\pard\li720{}{\f1{}request}\par
\pard\li1440{}The field {\f1{}request} points to the current recoding request, but may be changed as needed between recoding calls, for example when there is a need to achieve the construction of a resulting text made up of many pieces, each being recoded differently.\par
\par
\pard\li720{}{\f1{}input.name}\par
\pard\li720{}{\f1{}input.file}\par
\pard\li1440{}If {\f1{}input.name} is not {\f1{}NULL} at start of a recoding, this is a request that a file by that name be first opened for reading and later automatically closed once the whole file has been read. If the file name is not {\f1{}NULL} but an empty string, it means that standard input is to be used.  The opened file pointer is then held into {\f1{}input.file}.\par
\par
\pard\li1440{}If {\f1{}input.name} is {\f1{}NULL} and {\f1{}input.file} is not, than {\f1{}input.file} should point to a file already opened for read, which is meant to be recoded.\par
\par
\pard\li720{}{\f1{}input.buffer}\par
\pard\li720{}{\f1{}input.cursor}\par
\pard\li720{}{\f1{}input.limit}\par
\pard\li1440{}When both {\f1{}input.name} and {\f1{}input.file} are {\f1{}NULL}, three pointers describe an in-memory buffer containing the text to be recoded.  The buffer extends from {\f1{}input.buffer} to {\f1{}input.limit}, yet the text to be recoded only extends from {\f1{}input.cursor} to {\f1{}input.limit}.  In most situations, {\f1{}input.cursor} starts with the value that {\f1{}input.buffer} has.  (Its value will internally advance as the recoding goes, until it reaches the value of {\f1{}input.limit}.)\par
\par
\pard\li720{}{\f1{}output.name}\par
\pard\li720{}{\f1{}output.file}\par
\pard\li1440{}If {\f1{}output.name} is not {\f1{}NULL} at start of a recoding, this is a request that a file by that name be opened for write and later automatically closed after the recoding is done.  If the file name is not {\f1{}NULL} but an empty string, it means that standard output is to be used.  The opened file pointer is then held into {\f1{}output.file}.  If several passes with intermediate files are needed to produce the recoding, the {\f1{}output.name} file is opened only for the final pass.\par
\par
\pard\li1440{}If {\f1{}output.name} is {\f1{}NULL} and {\f1{}output.file} is not, then {\f1{}output.file} should point to a file already opened for write, which will receive the result of the recoding.\par
\par
\pard\li720{}{\f1{}output.buffer}\par
\pard\li720{}{\f1{}output.cursor}\par
\pard\li720{}{\f1{}output.limit}\par
\pard\li1440{}When both {\f1{}output.name} and {\f1{}output.file} are {\f1{}NULL}, three pointers describe an in-memory buffer meant to receive the text, once it is recoded.  The buffer is already allocated from {\f1{}output.buffer} to {\f1{}output.limit}.  In most situations, {\f1{}output.cursor} starts with the value that {\f1{}output.buffer} has.  Once the recoding is done, {\f1{}output.cursor} will point at the next free byte in the buffer, just after the recoded text, so another recoding could be called without changing any of these three pointers, for appending new information to it.  The number of recoded bytes in the buffer is the difference between {\f1{}output.cursor} and {\f1{}output.buffer}.\par
\par
\pard\li1440{}Each time {\f1{}output.cursor} reaches {\f1{}output.limit}, the buffer is reallocated bigger, possibly at a different location in memory, always held up-to-date in {\f1{}output.buffer}.  It is still possible to call a task level function with no output buffer at all to start with, in which case all three fields should have {\f1{}NULL} as a value.  This is the situation immediately after a call to {\f1{}recode_new_task}.\par
\par
\pard\li720{}{\f1{}strategy}\par
\pard\li1440{}This field, which is of type {\f1{}enum recode_sequence_strategy}, tells how various recoding steps (passes) will be interconnected.  Its initial value is {\f1{}RECODE_STRATEGY_UNDECIDED}, which is a constant defined in the header file {\f1{}<recodext.h>}.  Other possible values are:\par
\par
{\pard\li1440{}{\f1{}RECODE_SEQUENCE_IN_MEMORY}\par
\pard\li2160{}Keep intermediate recodings in memory.\par
\par
\pard\li1440{}{\f1{}RECODE_SEQUENCE_WITH_FILES}\par
\pard\li2160{}Do not fork, use intermediate files.\par
\par
\pard\li1440{}{\f1{}RECODE_SEQUENCE_WITH_PIPE}\par
\pard\li2160{}Fork processes connected with {\f1{}pipe(2)}.\par
\pard\li1440{}}\par
\pard\li1440{}The best for now is to leave this field alone, and let the recoding library decide its strategy, as many combinations have not been tested yet.\par
\par
\pard\li720{}{\f1{}byte_order_mark}\par
\pard\li1440{}This field, which is preset to {\f1{}true}, indicates that a byte order mark is to be expected at the beginning of any canonical {\f1{}UCS-2} or {\f1{}UTF-16} text, and that such a byte order mark should be also produced for these charsets.\par
\par
\pard\li720{}{\f1{}fail_level}\par
\pard\li1440{}This field, which is of type {\f1{}enum recode_error} (see {\uldb Errors}{\v Errors}), sets the error level at which task level functions should report a failure.  If an error being detected is equal or greater than {\f1{}fail_level}, the function will eventually return {\f1{}false} instead of {\f1{}true}.  The preset value for this field is {\f1{}RECODE_NOT_CANONICAL}, that means that if not reset to another value, the library will report failure on {\i any} error.\par
\par
\pard\li720{}{\f1{}abort_level}\par
\pard\li1440{}This field, which is of type {\f1{}enum recode_error} (see {\uldb Errors}{\v Errors}), sets the error level at which task level functions should immediately interrupt their processing.  If an error being detected is equal or greater than {\f1{}abort_level}, the function returns immediately, but the returned value ({\f1{}true} or {\f1{}false}) is still is decided from the setting of {\f1{}fail_level}, not {\f1{}abort_level}.  The preset value for this field is {\f1{}RECODE_MAXIMUM_ERROR}, that means that is not reset to another value, the library will never interrupt a recoding task.\par
\par
\pard\li720{}{\f1{}error_so_far}\par
\pard\li1440{}This field, which is of type {\f1{}enum recode_error} (see {\uldb Errors}{\v Errors}), maintains the maximum error level met so far while the recoding task was proceeding.  The preset value is {\f1{}RECODE_NO_ERROR}.\par
\par
\pard\li720{}}\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Task execution\par
\par
{\pard\keep\li1440\f1{}recode_perform_task ({\i task});\line
recode_filter_open ({\i task}, {\i file});\line
recode_filter_close ({\i task});\par
\pard\li720\f0{}}\par
\pard\li720{}The function {\f1{}recode_perform_task} reads as much input as possible, and recode all of it on prescribed output, given a properly initialised {\i task}.\par
\par
\pard\li720{}Functions {\f1{}recode_filter_open} and {\f1{}recode_filter_close} are only planned for now.  They are meant to read input in piecemeal ways.  Even if functionality already exists informally in the library, it has not been made available yet through such interface functions.\par
\par
\pard{}}\par
\page\pard Node: {\b Charset level}, \keepn Next: {\uldb Errors}{\v Errors}, Prev: {\uldb Task level}{\v Task_level}, Up: {\uldb Library}{\v Library}\line
K{\footnote K Charset level}
#{\footnote Charset_level}
${\footnote \pard{}Charset level functions}\par
\pard{\fs24\b Charset level functions}\par
\par
\pard{}Many functions are internal to the recoding library.  Some of them have been made external and available, for the {\f1{}recode} program had to retain all its previous functionality while being transformed into a mere application of the recoding library.  These functions are not really documented here for the time being, as we hope that many of them will vanish over time.  When this set of routines will stabilise, it would be convenient to document them as an API for handling charset names and contents.\par
\par
{\pard\keep\li720\f1{}RECODE_CHARSET find_charset ({\i name}, {\i cleaning-type});\line
bool list_all_charsets ({\i charset});\line
bool list_concise_charset ({\i charset}, {\i list-format});\line
bool list_full_charset ({\i charset});\par
\pard\f0{}}\par
\page\pard Node: {\b Errors}, \keepn Next: {\uldb }{\v }, Prev: {\uldb Charset level}{\v Charset_level}, Up: {\uldb Library}{\v Library}\line
K{\footnote K Errors}
#{\footnote Errors}
${\footnote \pard{}Handling errors}\par
\pard{\fs24\b Handling errors}\par
\par
\pard{}The {\f1{}recode} program, while using the {\f1{}recode} library, needs to control whether recoding problems are reported or not, and then reflect these in the exit status.  The program should also instruct the library about if the recoding should be abruptly interrupted when an error is met (so sparing processing when it is known in advance that a wrong result would be discarded anyway), or if it should proceed nevertheless.  Here is how the library groups errors into levels, listed here in order of increasing severity.\par
\par
{\pard{}{\f1{}RECODE_NO_ERROR}\par
\pard\li720{}No error was met on previous library calls.\par
\par
\pard{}{\f1{}RECODE_NOT_CANONICAL}\par
\pard\li720{}The input text was using one of the many alternative codings for some phenomenon, but not the one {\f1{}recode} would have canonically generated.  So, if the reverse recoding is later attempted, it would produce a text having the same {\i meaning} as the original text, yet not being byte identical.\par
\par
\pard\li720{}For example, a {\f1{}Base64} block in which end-of-lines appear elsewhere that at every 76 characters is not canonical.  An e-circumflex in TeX which is coded {\f1{}\'5C^\'7Be\'7D} instead of {\f1{}\'5C^e} is not canonical.\par
\par
\pard{}{\f1{}RECODE_AMBIGUOUS_OUTPUT}\par
\pard\li720{}It has been discovered that if the reverse recoding was attempted on the text output by this recoding, we would not obtain the original text, only because an ambiguity was generated by accident in the output text.  This ambiguity would then cause the wrong interpretation to be taken.\par
\par
\pard\li720{}Here are a few examples.  If the {\f1{}Latin-1} sequence {\f1{}e^} is converted to Easy French and back, the result will be interpreted as e-circumflex and so, will not reflect the intent of the original two characters.  Recoding an {\f1{}IBM-PC} text to {\f1{}Latin-1} and back, where the input text contained an isolated {\f1{}LF}, will have a spurious {\f1{}CR} inserted before the {\f1{}LF}.\par
\par
\pard\li720{}Currently, there are many cases in the library where the production of ambiguous output is not properly detected, as it is sometimes a difficult problem to accomplish this detection, or to do it speedily.\par
\par
\pard{}{\f1{}RECODE_UNTRANSLATABLE}\par
\pard\li720{}One or more input character could not be recoded, because there is just no representation for this character in the output charset.\par
\par
\pard\li720{}Here are a few examples.  Non-strict mode often allows {\f1{}recode} to compute on-the-fly mappings for unrepresentable characters, but strict mode prohibits such attribution of reversible translations: so strict mode might often trigger such an error.  Most {\f1{}UCS-2} codes used to represent Asian characters cannot be expressed in various Latin charsets.\par
\par
\pard{}{\f1{}RECODE_INVALID_INPUT}\par
\pard\li720{}The input text does comply with the coding it is declared to hold.  So, there is no way by which a reverse recoding would reproduce this text, because {\f1{}recode} should never produce invalid output.\par
\par
\pard\li720{}Here are a few examples.  In strict mode, {\f1{}ASCII} text just may not contain characters with the eight bit set.  {\f1{}UTF-8} encodings ought to be minimal({\ul 1}{\v Errors_1}).\par
\par
\pard{}{\f1{}RECODE_SYSTEM_ERROR}\par
\pard\li720{}The underlying system reported an error while the recoding was going on, likely an input/output error.  (This error symbol is currently unused in the library.)\par
\par
\pard{}{\f1{}RECODE_USER_ERROR}\par
\pard\li720{}The programmer or user requested something the recoding library is unable to provide, or used the API wrongly.  (This error symbol is currently unused in the library.)\par
\par
\pard{}{\f1{}RECODE_INTERNAL_ERROR}\par
\pard\li720{}Something really wrong, which should normally never happen, was detected within the recoding library.  This might be due to genuine bugs in the library, or maybe also, by un-initialised or overwritten arguments to the API.  (This error symbol is currently unused in the library.)\par
\par
\pard{}{\f1{}RECODE_MAXIMUM_ERROR}\par
\pard\li720{}This error code should never be returned, it is only internally used as a sentinel over all possible error codes.\par
\par
\pard{}}\par
\pard{}One should be able to set the error level threshold for returning failure at end of recoding, and also the threshold for immediate interruption.  If many errors occur while the recoding proceed, which are not severe enough to interrupt the recoding, then the most severe error is retained, while others are forgotten({\ul 2}{\v Errors_2}).  So, in case of an error, the possible actions currently are:\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}do nothing and let go, returning success at end of recoding,\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}just let go for now, but return failure at end of recoding,\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}interrupt recoding right away and return failure now.\par
\pard{}}\par
\pard{}See {\uldb Task level}{\v Task_level}, and particularly the description of fields {\f1{}fail_level}, {\f1{}abort_level} and {\f1{}error_so_far}, for more information about how errors are handled.\par
\par
\page\pard#{\footnote Errors_1}
\pard{}(1) The minimality of an {\f1{}UTF-8} encoding is guaranteed on output, but currently, it is not checked on input.\par
\par
\page\pard#{\footnote Errors_2}
\pard{}(2) Another approach would have been to define the level symbols as masks instead, and to give masks to threshold setting routines, and to retain all errors--yet I never met myself such a need in practice, and so, fear it would be overkill.  On the other hand, it might be interesting to maintain counters about how many times each kind of error occurred.\par
\par
\page\pard Node: {\b Universal}, \keepn Next: {\uldb Tabular}{\v Tabular}, Prev: {\uldb Library}{\v Library}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Universal}
#{\footnote Universal}
${\footnote \pard{}The universal charset}\par
\pard{\fs24\b The universal charset}\par
\par
\pard{}Standard ISO 10646 defines a universal character set, intended to encompass in the long run all languages written on this planet.  It is based on wide characters, and offer possibilities for two billion characters (2^31).\par
\par
\pard{}This charset was to become available in {\f1{}recode} under the name {\f1{}UCS}, with many external surfaces for it.  But in the current version, only surfaces of {\f1{}UCS} are offered, each presented as a genuine charset rather than a surface.  Such surfaces are only meaningful for the {\f1{}UCS} charset, so it is not that useful to draw a line between the surfaces and the only charset to which they may apply.\par
\par
{\f1{}\pard{}UCS} stands for Universal Character Set.  {\f1{}UCS-2} and {\f1{}UCS-4} are fixed length encodings, using two or four bytes per character respectively.  {\f1{}UTF} stands for {\f1{}UCS} Transformation Format, and are variable length encodings dedicated to {\f1{}UCS}.  {\f1{}UTF-1} was based on ISO 2022, it did not succeed({\ul 1}{\v Universal_1}).  {\f1{}UTF-2} replaced it, it has been called {\f1{}UTF-FSS} (File System Safe) in Unicode or Plan9 context, but is better known today as {\f1{}UTF-8}.  To complete the picture, there is {\f1{}UTF-16} based on 16 bits bytes, and {\f1{}UTF-7} which is meant for transmissions limited to 7-bit bytes.  Most often, one might see {\f1{}UTF-8} used for external storage, and {\f1{}UCS-2} used for internal storage.\par
\par
\pard{}When {\f1{}recode} is producing any representation of {\f1{}UCS}, it uses the replacement character {\f1{}U+FFFD} for any {\i valid} character which is not representable in the goal charset({\ul 2}{\v Universal_2}).  This happens, for example, when {\f1{}UCS-2} is not capable to echo a wide {\f1{}UCS-4} character, or for a similar reason, an {\f1{}UTF-8} sequence using more than three bytes.  The replacement character is meant to represent an existing character.  So, it is never produced to represent an invalid sequence or ill-formed character in the input text.  In such cases, {\f1{}recode} just get rid of the noise, while taking note of the error its usual ways.\par
\par
\pard{}Even if {\f1{}UTF-8} is an encoding, really, it is the encoding of a single character set, and nothing else.  It is useful to distinguish between an encoding (a {\i surface} within {\f1{}recode}) and a charset, but only when the surface may be applied to several charsets.  Specifying a charset is a bit simpler than specifying a surface in a {\f1{}recode} request.  There would not be a practical advantage at imposing a more complex syntax to {\f1{}recode} users, when it is simple to assimilate {\f1{}UTF-8} to a charset.  Similar considerations apply for {\f1{}UCS-2}, {\f1{}UCS-4}, {\f1{}UTF-16} and {\f1{}UTF-7}.  These are all considered to be charsets.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb UCS-2}{\v UCS_2}\tab Universal Character Set, 2 bytes\par
{\uldb UCS-4}{\v UCS_4}\tab Universal Character Set, 4 bytes\par
{\uldb UTF-7}{\v UTF_7}\tab Universal Transformation Format, 7 bits\par
{\uldb UTF-8}{\v UTF_8}\tab Universal Transformation Format, 8 bits\par
{\uldb UTF-16}{\v UTF_16}\tab Universal Transformation Format, 16 bits\par
{\uldb count-characters}{\v count_characters}\tab Frequency count of characters\par
{\uldb dump-with-names}{\v dump_with_names}\tab Fully interpreted UCS dump\par
\pard{}}\par
\page\pard#{\footnote Universal_1}
\pard{}(1) It is not probable that {\f1{}recode} will support {\f1{}UTF-1} ever.\par
\par
\page\pard#{\footnote Universal_2}
\pard{}(2) This is when the goal charset allows for 16-bits.  For shorter charsets, the {\f1{}--strict} ({\f1{}-s}) option decides what happens: either the character is dropped, or a reversible mapping is produced on the fly.\par
\par
\page\pard Node: {\b UCS-2}, \keepn Next: {\uldb UCS-4}{\v UCS_4}, Prev: {\uldb Universal}{\v Universal}, Up: {\uldb Universal}{\v Universal}\line
K{\footnote K UCS-2}
#{\footnote UCS_2}
${\footnote \pard{}Universal Character Set, 2 bytes}\par
\pard{\fs24\b Universal Character Set, 2 bytes}\par
\par
\pard{}One surface of {\f1{}UCS} is usable for the subset defined by its first sixty thousand characters (in fact, 31 * 2^11 codes), and uses exactly two bytes per character.  It is a mere dump of the internal memory representation which is {\i natural} for this subset and as such, conveys with it endianness problems.\par
\par
\pard{}A non-empty {\f1{}UCS-2} file normally begins with a so called \'A2byte order mark\'A2, having value {\f1{}0xFEFF}.  The value {\f1{}0xFFFE} is not an {\f1{}UCS} character, so if this value is seen at the beginning of a file, {\f1{}recode} reacts by swapping all pairs of bytes.  The library also properly reacts to other occurrences of {\f1{}0xFEFF} or {\f1{}0xFFFE} elsewhere than at the beginning, because concatenation of {\f1{}UCS-2} files should stay a simple matter, but it might trigger a diagnostic about non canonical input.\par
\par
\pard{}By default, when producing an {\f1{}UCS-2} file, {\f1{}recode} always output the high order byte before the low order byte.  But this could be easily overridden through the {\f1{}21-Permutation} surface.  For example, the command:\par
\par
{\pard\keep\li720\f1{}recode u8..u2/21 < {\i input} > {\i output}\par
\pard\f0{}}\par
\pard{}asks for an {\f1{}UTF-8} to {\f1{}UCS-2} conversion, with swapped byte output.\par
\par
\pard{}Use it as a genuine charset.  This charset is available in {\f1{}recode} under the name {\f1{}ISO-10646-UCS-2}.  Accepted aliases are {\f1{}UCS-2}, {\f1{}BMP}, {\f1{}rune} and {\f1{}u2}.\par
\par
\pard{}The {\f1{}recode} library is able to combine {\f1{}UCS-2} some sequences of codes into single code characters, to represent a few diacriticized characters, ligatures or diphtongs which have been included to ease mapping with other existing charsets.  It is also able to explode such single code characters into the corresponding sequence of codes.  The request syntax for triggering such operations is rudimentary and temporary.  The {\f1{}combined-UCS-2} pseudo character set is a special form of {\f1{}UCS-2} in which known combinings have been replaced by the simpler code.  Using {\f1{}combined-UCS-2} instead of {\f1{}UCS-2} in an {\i after} position of a request forces a combining step, while using {\f1{}combined-UCS-2} instead of {\f1{}UCS-2} in a {\i before} position of a request forces an exploding step.  For the time being, one has to resort to advanced request syntax to achieve other effects.  For example:\par
\par
{\pard\keep\li720\f1{}recode u8..co,u2..u8 < {\i input} > {\i output}\par
\pard\f0{}}\par
\pard{}copies an {\f1{}UTF-8} {\i input} over {\i output}, still to be in {\f1{}UTF-8}, yet merging combining characters into single codes whenever possible.\par
\par
\page\pard Node: {\b UCS-4}, \keepn Next: {\uldb UTF-7}{\v UTF_7}, Prev: {\uldb UCS-2}{\v UCS_2}, Up: {\uldb Universal}{\v Universal}\line
K{\footnote K UCS-4}
#{\footnote UCS_4}
${\footnote \pard{}Universal Character Set, 4 bytes}\par
\pard{\fs24\b Universal Character Set, 4 bytes}\par
\par
\pard{}Another surface of {\f1{}UCS} uses exactly four bytes per character, and is a mere dump of the internal memory representation which is {\i natural} for the whole charset and as such, conveys with it endianness problems.\par
\par
\pard{}Use it as a genuine charset.  This charset is available in {\f1{}recode} under the name {\f1{}ISO-10646-UCS-4}.  Accepted aliases are {\f1{}UCS}, {\f1{}UCS-4}, {\f1{}ISO_10646}, {\f1{}10646} and {\f1{}u4}.\par
\par
\page\pard Node: {\b UTF-7}, \keepn Next: {\uldb UTF-8}{\v UTF_8}, Prev: {\uldb UCS-4}{\v UCS_4}, Up: {\uldb Universal}{\v Universal}\line
K{\footnote K UTF-7}
#{\footnote UTF_7}
${\footnote \pard{}Universal Transformation Format, 7 bits}\par
\pard{\fs24\b Universal Transformation Format, 7 bits}\par
\par
{\f1{}\pard{}UTF-7} comes from IETF rather than ISO, and is described by RFC 1642, in the MIME series.  The {\f1{}UTF-7} encoding is meant to fit {\f1{}UCS-2} over channels limited to seven bits per byte.  It proceeds from a mix between the spirit of {\f1{}Quoted-Printable} and methods of {\f1{}Base64}, adapted to Unicode contexts.\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}UNICODE-1-1-UTF-7}.  Accepted aliases are {\f1{}UTF-7}, {\f1{}TF-7} and {\f1{}u7}.\par
\par
\page\pard Node: {\b UTF-8}, \keepn Next: {\uldb UTF-16}{\v UTF_16}, Prev: {\uldb UTF-7}{\v UTF_7}, Up: {\uldb Universal}{\v Universal}\line
K{\footnote K UTF-8}
#{\footnote UTF_8}
${\footnote \pard{}Universal Transformation Format, 8 bits}\par
\pard{\fs24\b Universal Transformation Format, 8 bits}\par
\par
\pard{}Even if {\f1{}UTF-8} does not originally come from IETF, there is now RFC 2279 to describe it.  In letters sent on 1995-01-21 and 1995-04-20, Markus Kuhn writes:\par
\par
{{\f1{}\pard\li720{}UTF-8} is an {\f1{}ASCII} compatible multi-byte encoding of the ISO 10646 universal character set ({\f1{}UCS}).  {\f1{}UCS} is a 31-bit superset of all other character set standards.  The first 256 characters of {\f1{}UCS} are identical to those of ISO 8859-1 (Latin-1).  The {\f1{}UCS-2} encoding of UCS is a sequence of bigendian 16-bit words, the {\f1{}UCS-4} encoding is a sequence of bigendian 32-bit words.  The {\f1{}UCS-2} subset of ISO 10646 is also known as "Unicode".  As both {\f1{}UCS-2} and {\f1{}UCS-4} require heavy modifications to traditional {\f1{}ASCII} oriented system designs (e.g. Unix), the {\f1{}UTF-8} encoding has been designed for these applications.\par
\par
\pard\li720{}In {\f1{}UTF-8}, only {\f1{}ASCII} characters are encoded using bytes below 128.  All other non-ASCII characters are encoded as multi-byte sequences consisting only of bytes in the range 128-253.  This avoids critical bytes like {\f1{}NUL} and {\f1{}/} in {\f1{}UTF-8} strings, which makes the {\f1{}UTF-8} encoding suitable for being handled by the standard C string library and being used in Unix file names.  Other properties include the preserved lexical sorting order and that {\f1{}UTF-8} allows easy self-synchronisation of software receiving {\f1{}UTF-8} strings.\par
\pard{}}\par
{\f1{}\pard{}UTF-8} is the most common external surface of {\f1{}UCS}, each character uses from one to six bytes, and is able to encode all 2^31 characters of the {\f1{}UCS}.  It is implemented as a charset, with the following properties.\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Strict 7-bit {\f1{}ASCII} is completely invariant under {\f1{}UTF-8}, and those are the only one-byte characters.  {\f1{}UCS} values and {\f1{}ASCII} values coincide.  No multi-byte characters ever contain bytes less than 128.  {\f1{}NUL} {\i is} {\f1{}NUL}.  A multi-byte character always starts with a byte of 192 or more, and is always followed by a number of bytes between 128 to 191.  That means that you may read at random on disk or memory, and easily discover the start of the current, next or previous character.  You can count, skip or extract characters with this only knowledge.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}If you read the first byte of a multi-byte character in binary, it contains many {\f1{}1} bits in successions starting with the most significant one (from the left), at least two.  The length of this {\f1{}1} sequence equals the byte size of the character.  All succeeding bytes start by {\f1{}10}.  This is a lot of redundancy, making it fairly easy to guess that a file is valid {\f1{}UTF-8}, or to safely state that it is not.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}In a multi-byte character, if you remove all leading {\f1{}1} bits of the first byte of a multi-byte character, and the initial {\f1{}10} bits of all remaining bytes (so keeping 6 bits per byte for those), the remaining bits concatenated are the UCS value.\par
\par
\pard{}}\par
\pard{}These properties also have a few nice consequences.\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Conversion to/from values is algorithmically simple, and reasonably speedy.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}A sequence of {\i N} bytes can hold characters needing up to 2 + 5{\i N} bits in their {\f1{}UCS} representation.  Here, {\i N} is a number between 1 and 6.  So, {\f1{}UTF-8} is most economical when mapping ASCII (1 byte), then {\f1{}UCS-2} (1 to 3 bytes), then {\f1{}UCS-4} (1 to 6 bytes).\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}The lexicographic sorting order of {\f1{}UCS} strings is preserved.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Bytes with value 254 or 255 never appear, and because of that, these are sometimes used when escape mechanisms are needed.\par
\par
\pard{}}\par
\pard{}In some case, when little processing is done on a lot of strings, one may choose for efficiency reasons to handle {\f1{}UTF-8} strings directly even if variable length, as it is easy to get start of characters.  Character insertion or replacement might require moving the remainder of the string in either direction.  In most cases, it is faster and easier to convert from {\f1{}UTF-8} to {\f1{}UCS-2} or {\f1{}UCS-4} prior to processing.\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}UTF-8}.  Accepted aliases are {\f1{}UTF-2}, {\f1{}UTF-FSS}, {\f1{}FSS_UTF}, {\f1{}TF-8} and {\f1{}u8}.\par
\par
\page\pard Node: {\b UTF-16}, \keepn Next: {\uldb count-characters}{\v count_characters}, Prev: {\uldb UTF-8}{\v UTF_8}, Up: {\uldb Universal}{\v Universal}\line
K{\footnote K UTF-16}
#{\footnote UTF_16}
${\footnote \pard{}Universal Transformation Format, 16 bits}\par
\pard{\fs24\b Universal Transformation Format, 16 bits}\par
\par
\pard{}Another external surface of {\f1{}UCS} is also variable length, each character using either two or four bytes.  It is usable for the subset defined by the first million characters (17 * 2^16) of {\f1{}UCS}.\par
\par
\pard{}Martin J. Du"rst writes (to {\f1{}comp.std.internat}, on 1995-03-28):\par
\par
{{\f1{}\pard\li720{}UTF-16} is another method that reserves two times 1024 codepoints in Unicode and uses them to index around one million additional characters.  {\f1{}UTF-16} is a little bit like former multibyte codes, but quite not so, as both the first and the second 16-bit code clearly show what they are.  The idea is that one million codepoints should be enough for all the rare Chinese ideograms and historical scripts that do not fit into the Base Multilingual Plane of ISO 10646 (with just about 63,000 positions available, now that 2,000 are gone).\par
\pard{}}\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}UTF-16}.  Accepted aliases are {\f1{}Unicode}, {\f1{}TF-16} and {\f1{}u6}.\par
\par
\page\pard Node: {\b count-characters}, \keepn Next: {\uldb dump-with-names}{\v dump_with_names}, Prev: {\uldb UTF-16}{\v UTF_16}, Up: {\uldb Universal}{\v Universal}\line
K{\footnote K count-characters}
#{\footnote count_characters}
${\footnote \pard{}Frequency count of characters}\par
\pard{\fs24\b Frequency count of characters}\par
\par
\pard{}A device may be used to obtain a list of characters in a file, and how many times each character appears.  Each count is followed by the {\f1{}UCS-2} value of the character and, when known, the RFC 1345 mnemonic for that character.\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}count-characters}.\par
\par
\pard{}This {\f1{}count} feature has been implemented as a charset.  This may change in some later version, as it would sometimes be convenient to count original bytes, instead of their {\f1{}UCS-2} equivalent.\par
\par
\page\pard Node: {\b dump-with-names}, \keepn Next: {\uldb }{\v }, Prev: {\uldb count-characters}{\v count_characters}, Up: {\uldb Universal}{\v Universal}\line
K{\footnote K dump-with-names}
#{\footnote dump_with_names}
${\footnote \pard{}Fully interpreted UCS dump}\par
\pard{\fs24\b Fully interpreted UCS dump}\par
\par
\pard{}Another device may be used to get fully interpreted dumps of an {\f1{}UCS-2} stream of characters, with one {\f1{}UCS-2} character displayed on a full output line.  Each line receives the RFC 1345 mnemonic for the character if it exists, the {\f1{}UCS-2} value of the character, and a descriptive comment for that character.  As each input character produces its own output line, beware that the output file from this conversion may be much, much bigger than the input file.\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}dump-with-names}.\par
\par
\pard{}This {\f1{}dump-with-names} feature has been implemented as a charset rather than a surface.  This is surely debatable.  The current implementation allows for dumping charsets other than {\f1{}UCS-2}.  For example, the command {\f1{}recode l2..full < {\i input}} implies a necessary conversion from {\f1{}Latin-2} to {\f1{}UCS-2}, as {\f1{}full-dump} is only connected out from {\f1{}UCS-2}.  In such cases, {\f1{}recode} does not display the original {\f1{}Latin-2} codes in the dump, only the corresponding {\f1{}UCS-2} values.  To give a simpler example, the command\par
\par
{\pard\keep\li720\f1{}echo 'Hello, world!' | recode us..dump\par
\pard\f0{}}\par
\pard{}produces the following output.\par
\par
{\pard\keep\li720\f1{}Mne   UCS2   Description\line
\line
H     0048   latin capital letter h\line
e     0065   latin small letter e\line
l     006C   latin small letter l\line
l     006C   latin small letter l\line
o     006F   latin small letter o\line
,     002C   comma\line
SP    0020   space\line
w     0077   latin small letter w\line
o     006F   latin small letter o\line
r     0072   latin small letter r\line
l     006C   latin small letter l\line
d     0064   latin small letter d\line
!     0021   exclamation mark\line
LF    000A   line feed (lf)\par
\pard\f0{}}\par
\pard{}The descriptive comment is given in English and {\f1{}ASCII}, yet if the English description is not available but a French one is, then the French description is given instead, using {\f1{}Latin-1}.  However, if the {\f1{}LANGUAGE} or {\f1{}LANG} environment variable begins with the letters {\f1{}fr}, then listing preference goes to French when both descriptions are available.\par
\par
\pard{}Here is another example.  To get the long description of the code 237 in Latin-5 table, one may use the following command.\par
\par
{\pard\keep\li720\f1{}echo -n 237 | recode l5/d..dump\par
\pard\f0{}}\par
\pard{}If your {\f1{}echo} does not grok {\f1{}-n}, use {\f1{}echo 237\'5Cc} instead.  Here is how to see what Unicode {\f1{}U+03C6} means, while getting rid of the title lines.\par
\par
{\pard\keep\li720\f1{}echo -n 0x03C6 | recode u2/x2..dump | tail +3\par
\pard\f0{}}\par
\page\pard Node: {\b Tabular}, \keepn Next: {\uldb ASCII misc}{\v ASCII_misc}, Prev: {\uldb Universal}{\v Universal}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Tabular}
#{\footnote Tabular}
${\footnote \pard{}Tabular sources (RFC 1345)}\par
\pard{\fs24\b Tabular sources (RFC 1345)}\par
\par
\pard{}An important part of the tabular charset knowledge in {\f1{}recode} comes from RFC 1345 or, alternatively, from the {\f1{}chset} tools, both maintained by Keld Simonsen.  The RFC 1345 document:\par
\par
{\pard\li720{}"Character Mnemonics & Character Sets", K. Simonsen, Request for Comments no. 1345, Network Working Group, June 1992.\par
\pard{}}\par
\pard{}defines many character mnemonics and character sets.  The {\f1{}recode} library implements most of RFC 1345, however:\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}It does not recognise those charsets which overload character positions: {\f1{}dk-us} and {\f1{}us-dk}.  However, see {\uldb Mixed}{\v Mixed}.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}It does not recognise those charsets which combine two characters for representing a third: {\f1{}ANSI_X3.110-1983}, {\f1{}ISO_6937-2-add}, {\f1{}T.101-G2}, {\f1{}T.61-8bit}, {\f1{}iso-ir-90} and {\f1{}videotex-suppl}.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}It does not recognise 16-bits charsets: {\f1{}GB_2312-80}, {\f1{}JIS_C6226-1978}, {\f1{}JIS_C6226-1983}, {\f1{}JIS_X0212-1990} and {\f1{}KS_C_5601-1987}.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}It interprets the charset {\f1{}isoir91} as {\f1{}NATS-DANO} (alias {\f1{}iso-ir-9-1}), {\i not} as {\f1{}JIS_C6229-1984-a} (alias {\f1{}iso-ir-91}).  It also interprets the charset {\f1{}isoir92} as {\f1{}NATS-DANO-ADD} (alias {\f1{}iso-ir-9-2}), {\i not} as {\f1{}JIS_C6229-1984-b} (alias {\f1{}iso-ir-92}).  It might be better just avoiding these two alias names.\par
\par
\pard{}}\par
\pard{}Keld Simonsen <keld@dkuug.dk> did most of RFC 1345 himself, with some funding from Danish Standards and Nordic standards (INSTA) project.  He also did the character set design work, with substantial input from Olle Jaernefors.  Keld typed in almost all of the tables, some have been contributed.  A number of people have checked the tables in various ways.  The RFC lists a number of people who helped.\par
\par
\pard{}Keld and the {\f1{}recode} maintainer have an arrangement by which any new discovered information submitted by {\f1{}recode} users, about tabular charsets, is forwarded to Keld, eventually merged into Keld's work, and only then, reimported into {\f1{}recode}.  Neither the {\f1{}recode} program nor its library try to compete, nor even establish themselves as an alternate or diverging reference: RFC 1345 and its new drafts stay the genuine source for most tabular information conveyed by {\f1{}recode}.  Keld has been more than collaborative so far, so there is no reason that we act otherwise.  In a word, {\f1{}recode} should be perceived as the application of external references, but not as a reference in itself.\par
\par
\pard{}Internally, RFC 1345 associates which each character an unambiguous mnemonic of a few characters, taken from ISO 646, which is a minimal ASCII subset of 83 characters.  The charset made up by these mnemonics is available in {\f1{}recode} under the name {\f1{}RFC1345}.  It has {\f1{}mnemonic} and {\f1{}1345} for aliases.  As implemened, this charset exactly corresponds to {\f1{}mnemonic+ascii+38}, using RFC 1345 nomenclature.  Roughly said, ISO 646 characters represent themselves, except for the ampersand ({\f1{}&}) which appears doubled.  A prefix of a single ampersand introduces a mnemonic.  For mnemonics using two characters, the prefix is immediately by the mnemonic.  For longer mnemonics, the prefix is followed by an underline ({\f1{}_}), the mmemonic, and another underline.  Conversions to this charset are usually reversible.\par
\par
\pard{}Currently, {\f1{}recode} does not offer any of the many other possible variations of this family of representations.  They will likely be implemented in some future version, however.\par
\par
{\pard{}{\f1{}ANSI_X3.4-1968}\par
{\f1{}\pard\li720{}367}, {\f1{}ANSI_X3.4-1986}, {\f1{}ASCII}, {\f1{}CP367}, {\f1{}IBM367}, {\f1{}ISO646-US}, {\f1{}ISO_646.irv:1991}, {\f1{}US-ASCII}, {\f1{}iso-ir-6} and {\f1{}us} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ASMO_449}\par
{\f1{}\pard\li720{}ISO_9036}, {\f1{}arabic7} and {\f1{}iso-ir-89} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}BS_4730}\par
{\f1{}\pard\li720{}ISO646-GB}, {\f1{}gb}, {\f1{}iso-ir-4} and {\f1{}uk} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}BS_viewdata}\par
{\f1{}\pard\li720{}iso-ir-47} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}CP1250}\par
{\f1{}\pard\li720{}1250}, {\f1{}ms-ee} and {\f1{}windows-1250} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}CP1251}\par
{\f1{}\pard\li720{}1251}, {\f1{}ms-cyrl} and {\f1{}windows-1251} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}CP1252}\par
{\f1{}\pard\li720{}1252}, {\f1{}ms-ansi} and {\f1{}windows-1252} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}CP1253}\par
{\f1{}\pard\li720{}1253}, {\f1{}ms-greek} and {\f1{}windows-1253} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}CP1254}\par
{\f1{}\pard\li720{}1254}, {\f1{}ms-turk} and {\f1{}windows-1254} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}CP1255}\par
{\f1{}\pard\li720{}1255}, {\f1{}ms-hebr} and {\f1{}windows-1255} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}CP1256}\par
{\f1{}\pard\li720{}1256}, {\f1{}ms-arab} and {\f1{}windows-1256} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}CP1257}\par
{\f1{}\pard\li720{}1257}, {\f1{}WinBaltRim} and {\f1{}windows-1257} are aliases for this charset.  Source: CEN/TC304 N283.\par
\par
\pard{}{\f1{}CSA_Z243.4-1985-1}\par
{\f1{}\pard\li720{}ISO646-CA}, {\f1{}ca}, {\f1{}csa7-1} and {\f1{}iso-ir-121} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}CSA_Z243.4-1985-2}\par
{\f1{}\pard\li720{}ISO646-CA2}, {\f1{}csa7-2} and {\f1{}iso-ir-122} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}CSA_Z243.4-1985-gr}\par
{\f1{}\pard\li720{}iso-ir-123} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}CSN_369103}\par
{\f1{}\pard\li720{}KOI-8_L2}, {\f1{}iso-ir-139} and {\f1{}koi8l2} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}CWI}\par
{\f1{}\pard\li720{}CWI-2} and {\f1{}cp-hu} are aliases for this charset.  Source: Computerworld Sza'mita'stechnika vol 3 issue 13 1988-06-29.\par
\par
\pard{}{\f1{}DEC-MCS}\par
{\f1{}\pard\li720{}dec} is an alias for this charset.  VAX/VMS User's Manual, Order Number: AI-Y517A-TE, April 1986.\par
\par
\pard{}{\f1{}DIN_66003}\par
{\f1{}\pard\li720{}ISO646-DE}, {\f1{}de} and {\f1{}iso-ir-21} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}DS_2089}\par
{\f1{}\pard\li720{}DS2089}, {\f1{}ISO646-DK} and {\f1{}dk} are aliases for this charset.  Source: Danish Standard, DS 2089, February 1974.\par
\par
\pard{}{\f1{}EBCDIC-AT-DE}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-AT-DE-A}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-CA-FR}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-DK-NO}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-DK-NO-A}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-ES}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-ES-A}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-ES-S}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-FI-SE}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-FI-SE-A}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-FR}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-IS-FRISS}\par
{\f1{}\pard\li720{}friss} is an alias for this charset.  Source: Skyrsuvelar Rikisins og Reykjavikurborgar, feb 1982.\par
\par
\pard{}{\f1{}EBCDIC-IT}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-PT}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-UK}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}EBCDIC-US}\par
\pard\li720{}Source: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\par
\par
\pard{}{\f1{}ECMA-cyrillic}\par
{\f1{}\pard\li720{}ECMA-113}, {\f1{}ECMA-113:1986} and {\f1{}iso-ir-111} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ES}\par
{\f1{}\pard\li720{}ISO646-ES} and {\f1{}iso-ir-17} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ES2}\par
{\f1{}\pard\li720{}ISO646-ES2} and {\f1{}iso-ir-85} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}GB_1988-80}\par
{\f1{}\pard\li720{}ISO646-CN}, {\f1{}cn} and {\f1{}iso-ir-57} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}GOST_19768-87}\par
{\f1{}\pard\li720{}ST_SEV_358-88} and {\f1{}iso-ir-153} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}IBM037}\par
{\f1{}\pard\li720{}037}, {\f1{}CP037}, {\f1{}ebcdic-cp-ca}, {\f1{}ebcdic-cp-nl}, {\f1{}ebcdic-cp-us} and {\f1{}ebcdic-cp-wt} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM038}\par
{\f1{}\pard\li720{}038}, {\f1{}CP038} and {\f1{}EBCDIC-INT} are aliases for this charset.  Source: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\par
\par
\pard{}{\f1{}IBM1004}\par
{\f1{}\pard\li720{}1004}, {\f1{}CP1004} and {\f1{}os2latin1} are aliases for this charset.  Source: CEN/TC304 N283, 1994-02-04.\par
\par
\pard{}{\f1{}IBM1026}\par
{\f1{}\pard\li720{}1026} and {\f1{}CP1026} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM1047}\par
{\f1{}\pard\li720{}1047} and {\f1{}CP1047} are aliases for this charset.  Source: IBM Character Data Representation Architecture.  Registry SC09-1391-00 p 150.\par
\par
\pard{}{\f1{}IBM256}\par
{\f1{}\pard\li720{}256}, {\f1{}CP256} and {\f1{}EBCDIC-INT1} are aliases for this charset.  Source: IBM Registry C-H 3-3220-050.\par
\par
\pard{}{\f1{}IBM273}\par
{\f1{}\pard\li720{}273} and {\f1{}CP273} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM274}\par
{\f1{}\pard\li720{}274}, {\f1{}CP274} and {\f1{}EBCDIC-BE} are aliases for this charset.  Source: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\par
\par
\pard{}{\f1{}IBM275}\par
{\f1{}\pard\li720{}275}, {\f1{}CP275} and {\f1{}EBCDIC-BR} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM277}\par
{\f1{}\pard\li720{}EBCDIC-CP-DK} and {\f1{}EBCDIC-CP-NO} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM278}\par
{\f1{}\pard\li720{}278}, {\f1{}CP278}, {\f1{}ebcdic-cp-fi} and {\f1{}ebcdic-cp-se} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM280}\par
{\f1{}\pard\li720{}280}, {\f1{}CP280} and {\f1{}ebcdic-cp-it} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM281}\par
{\f1{}\pard\li720{}281}, {\f1{}CP281} and {\f1{}EBCDIC-JP-E} are aliases for this charset.  Source: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\par
\par
\pard{}{\f1{}IBM284}\par
{\f1{}\pard\li720{}284}, {\f1{}CP284} and {\f1{}ebcdic-cp-es} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM285}\par
{\f1{}\pard\li720{}285}, {\f1{}CP285} and {\f1{}ebcdic-cp-gb} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM290}\par
{\f1{}\pard\li720{}290}, {\f1{}CP290} and {\f1{}EBCDIC-JP-kana} are aliases for this charset.  Source: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\par
\par
\pard{}{\f1{}IBM297}\par
{\f1{}\pard\li720{}297}, {\f1{}CP297} and {\f1{}ebcdic-cp-fr} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM420}\par
{\f1{}\pard\li720{}420}, {\f1{}CP420} and {\f1{}ebcdic-cp-ar1} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.  IBM NLS RM p 11-11.\par
\par
\pard{}{\f1{}IBM423}\par
{\f1{}\pard\li720{}423}, {\f1{}CP423} and {\f1{}ebcdic-cp-gr} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM424}\par
{\f1{}\pard\li720{}424}, {\f1{}CP424} and {\f1{}ebcdic-cp-he} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM437}\par
{\f1{}\pard\li720{}437} and {\f1{}CP437} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM500}\par
{\f1{}\pard\li720{}500}, {\f1{}500V1}, {\f1{}CP500}, {\f1{}ebcdic-cp-be} and {\f1{}ebcdic-cp-ch} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM850}\par
{\f1{}\pard\li720{}850} and {\f1{}CP850} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}IBM851}\par
{\f1{}\pard\li720{}851} and {\f1{}CP851} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM852}\par
{\f1{}\pard\li720{}852}, {\f1{}CP852}, {\f1{}pcl2} and {\f1{}pclatin2} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM855}\par
{\f1{}\pard\li720{}855} and {\f1{}CP855} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM857}\par
{\f1{}\pard\li720{}857} and {\f1{}CP857} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM860}\par
{\f1{}\pard\li720{}860} and {\f1{}CP860} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM861}\par
{\f1{}\pard\li720{}861}, {\f1{}CP861} and {\f1{}cp-is} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM862}\par
{\f1{}\pard\li720{}862} and {\f1{}CP862} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM863}\par
{\f1{}\pard\li720{}863} and {\f1{}CP863} are aliases for this charset.  Source: IBM Keyboard layouts and code pages, PN 07G4586 June 1991.\par
\par
\pard{}{\f1{}IBM864}\par
{\f1{}\pard\li720{}864} and {\f1{}CP864} are aliases for this charset.  Source: IBM Keyboard layouts and code pages, PN 07G4586 June 1991.\par
\par
\pard{}{\f1{}IBM865}\par
{\f1{}\pard\li720{}865} and {\f1{}CP865} are aliases for this charset.  Source: IBM DOS 3.3 Ref (Abridged), 94X9575 (Feb 1987).\par
\par
\pard{}{\f1{}IBM868}\par
{\f1{}\pard\li720{}868}, {\f1{}CP868} and {\f1{}cp-ar} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM869}\par
{\f1{}\pard\li720{}869}, {\f1{}CP869} and {\f1{}cp-gr} are aliases for this charset.  Source: IBM Keyboard layouts and code pages, PN 07G4586 June 1991.\par
\par
\pard{}{\f1{}IBM870}\par
{\f1{}\pard\li720{}870}, {\f1{}CP870}, {\f1{}ebcdic-cp-roece} and {\f1{}ebcdic-cp-yu} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM871}\par
{\f1{}\pard\li720{}871}, {\f1{}CP871} and {\f1{}ebcdic-cp-is} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM875}\par
{\f1{}\pard\li720{}875}, {\f1{}CP875} and {\f1{}EBCDIC-Greek} are aliases for this charset.  Source: UNICODE 1.0.\par
\par
\pard{}{\f1{}IBM880}\par
{\f1{}\pard\li720{}880}, {\f1{}CP880} and {\f1{}EBCDIC-Cyrillic} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM891}\par
{\f1{}\pard\li720{}891} and {\f1{}CP891} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM903}\par
{\f1{}\pard\li720{}903} and {\f1{}CP903} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM904}\par
{\f1{}\pard\li720{}904} and {\f1{}CP904} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IBM905}\par
{\f1{}\pard\li720{}905}, {\f1{}CP905} and {\f1{}ebcdic-cp-tr} are aliases for this charset.  Source: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\par
\par
\pard{}{\f1{}IBM918}\par
{\f1{}\pard\li720{}918}, {\f1{}CP918} and {\f1{}ebcdic-cp-ar2} are aliases for this charset.  Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\par
\par
\pard{}{\f1{}IEC_P27-1}\par
{\f1{}\pard\li720{}iso-ir-143} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}INIS}\par
{\f1{}\pard\li720{}iso-ir-49} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}INIS-8}\par
{\f1{}\pard\li720{}iso-ir-50} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}INIS-cyrillic}\par
{\f1{}\pard\li720{}iso-ir-51} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}INVARIANT}\par
{\f1{}\pard\li720{}iso-ir-170} is an alias for this charset.\par
\par
\pard{}{\f1{}ISO-8859-1}\par
{\f1{}\pard\li720{}819}, {\f1{}CP819}, {\f1{}IBM819}, {\f1{}ISO_8859-1}, {\f1{}ISO_8859-1:1987}, {\f1{}iso-ir-100}, {\f1{}l1} and {\f1{}latin1} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-10}\par
{\f1{}\pard\li720{}ISO_8859-10}, {\f1{}ISO_8859-10:1993}, {\f1{}iso-ir-157}, {\f1{}l6} and {\f1{}latin6} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-13}\par
{\f1{}\pard\li720{}ISO_8859-13}, {\f1{}ISO_8859-13:1998}, {\f1{}iso-baltic}, {\f1{}iso-ir-179a}, {\f1{}l7} and {\f1{}latin7} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-14}\par
{\f1{}\pard\li720{}ISO_8859-14}, {\f1{}ISO_8859-14:1998}, {\f1{}iso-celtic}, {\f1{}iso-ir-199}, {\f1{}l8} and {\f1{}latin8} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-15}\par
{\f1{}\pard\li720{}ISO_8859-15}, {\f1{}ISO_8859-15:1998}, {\f1{}iso-ir-203}, {\f1{}l9} and {\f1{}latin9} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-2}\par
{\f1{}\pard\li720{}912}, {\f1{}CP912}, {\f1{}IBM912}, {\f1{}ISO_8859-2}, {\f1{}ISO_8859-2:1987}, {\f1{}iso-ir-101}, {\f1{}l2} and {\f1{}latin2} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-3}\par
{\f1{}\pard\li720{}ISO_8859-3}, {\f1{}ISO_8859-3:1988}, {\f1{}iso-ir-109}, {\f1{}l3} and {\f1{}latin3} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-4}\par
{\f1{}\pard\li720{}ISO_8859-4}, {\f1{}ISO_8859-4:1988}, {\f1{}iso-ir-110}, {\f1{}l4} and {\f1{}latin4} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-5}\par
{\f1{}\pard\li720{}ISO_8859-5}, {\f1{}ISO_8859-5:1988}, {\f1{}cyrillic} and {\f1{}iso-ir-144} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-6}\par
{\f1{}\pard\li720{}ASMO-708}, {\f1{}ECMA-114}, {\f1{}ISO_8859-6}, {\f1{}ISO_8859-6:1987}, {\f1{}arabic} and {\f1{}iso-ir-127} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-7}\par
{\f1{}\pard\li720{}ECMA-118}, {\f1{}ELOT_928}, {\f1{}ISO_8859-7}, {\f1{}ISO_8859-7:1987}, {\f1{}greek}, {\f1{}greek8} and {\f1{}iso-ir-126} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-8}\par
{\f1{}\pard\li720{}ISO_8859-8}, {\f1{}ISO_8859-8:1988}, {\f1{}hebrew} and {\f1{}iso-ir-138} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO-8859-9}\par
{\f1{}\pard\li720{}ISO_8859-9}, {\f1{}ISO_8859-9:1989}, {\f1{}iso-ir-148}, {\f1{}l5} and {\f1{}latin5} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_10367-box}\par
{\f1{}\pard\li720{}iso-ir-155} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_2033-1983}\par
{\f1{}\pard\li720{}e13b} and {\f1{}iso-ir-98} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_5427}\par
{\f1{}\pard\li720{}iso-ir-37} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_5427-ext}\par
{\f1{}\pard\li720{}ISO_5427:1981} and {\f1{}iso-ir-54} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_5428}\par
{\f1{}\pard\li720{}ISO_5428:1980} and {\f1{}iso-ir-55} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_646.basic}\par
{\f1{}\pard\li720{}ISO_646.basic:1983} and {\f1{}ref} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_646.irv}\par
{\f1{}\pard\li720{}ISO_646.irv:1983}, {\f1{}irv} and {\f1{}iso-ir-2} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_6937-2-25}\par
{\f1{}\pard\li720{}iso-ir-152} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}ISO_8859-supp}\par
{\f1{}\pard\li720{}iso-ir-154} and {\f1{}latin1-2-5} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}IT}\par
{\f1{}\pard\li720{}ISO646-IT} and {\f1{}iso-ir-15} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6220-1969-jp}\par
{\f1{}\pard\li720{}JIS_C6220-1969}, {\f1{}iso-ir-13}, {\f1{}katakana} and {\f1{}x0201-7} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6220-1969-ro}\par
{\f1{}\pard\li720{}ISO646-JP}, {\f1{}iso-ir-14} and {\f1{}jp} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6229-1984-a}\par
{\f1{}\pard\li720{}jp-ocr-a} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6229-1984-b}\par
{\f1{}\pard\li720{}ISO646-JP-OCR-B} and {\f1{}jp-ocr-b} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6229-1984-b-add}\par
{\f1{}\pard\li720{}iso-ir-93} and {\f1{}jp-ocr-b-add} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6229-1984-hand}\par
{\f1{}\pard\li720{}iso-ir-94} and {\f1{}jp-ocr-hand} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6229-1984-hand-add}\par
{\f1{}\pard\li720{}iso-ir-95} and {\f1{}jp-ocr-hand-add} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_C6229-1984-kana}\par
{\f1{}\pard\li720{}iso-ir-96} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JIS_X0201}\par
{\f1{}\pard\li720{}X0201} is an alias for this charset.\par
\par
\pard{}{\f1{}JUS_I.B1.002}\par
{\f1{}\pard\li720{}ISO646-YU}, {\f1{}iso-ir-141}, {\f1{}js} and {\f1{}yu} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JUS_I.B1.003-mac}\par
{\f1{}\pard\li720{}iso-ir-147} and {\f1{}macedonian} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}JUS_I.B1.003-serb}\par
{\f1{}\pard\li720{}iso-ir-146} and {\f1{}serbian} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}KOI-7}\par
\pard\li720{}Source: Andrey A. Chernov <ache@nagual.pp.ru>.\par
\par
\pard{}{\f1{}KOI-8}\par
{\f1{}\pard\li720{}GOST_19768-74} is an alias for this charset.  Source: Andrey A. Chernov <ache@nagual.pp.ru>.\par
\par
\pard{}{\f1{}KOI8-R}\par
\pard\li720{}Source: RFC1489 via Gabor Kiss <kissg@sztaki.hu>.  And Andrey A. Chernov <ache@nagual.pp.ru>.\par
\par
\pard{}{\f1{}KOI8-RU}\par
\pard\li720{}Source: http://cad.ntu-kpi.kiev.ua/multiling/koi8-ru/.\par
\par
\pard{}{\f1{}KOI8-U}\par
\pard\li720{}Source: RFC 2319.  Mibenum: 2088.  Source: http://www.net.ua/KOI8-U/.\par
\par
\pard{}{\f1{}KSC5636}\par
{\f1{}\pard\li720{}ISO646-KR} is an alias for this charset.\par
\par
\pard{}{\f1{}Latin-greek-1}\par
{\f1{}\pard\li720{}iso-ir-27} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}MSZ_7795.3}\par
{\f1{}\pard\li720{}ISO646-HU}, {\f1{}hu} and {\f1{}iso-ir-86} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NATS-DANO}\par
{\f1{}\pard\li720{}iso-ir-9-1} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NATS-DANO-ADD}\par
{\f1{}\pard\li720{}iso-ir-9-2} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NATS-SEFI}\par
{\f1{}\pard\li720{}iso-ir-8-1} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NATS-SEFI-ADD}\par
{\f1{}\pard\li720{}iso-ir-8-2} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NC_NC00-10}\par
{\f1{}\pard\li720{}ISO646-CU}, {\f1{}NC_NC00-10:81}, {\f1{}cuba} and {\f1{}iso-ir-151} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NF_Z_62-010}\par
{\f1{}\pard\li720{}ISO646-FR}, {\f1{}fr} and {\f1{}iso-ir-69} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NF_Z_62-010_(1973)}\par
{\f1{}\pard\li720{}ISO646-FR1} and {\f1{}iso-ir-25} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NS_4551-1}\par
{\f1{}\pard\li720{}ISO646-NO}, {\f1{}iso-ir-60} and {\f1{}no} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NS_4551-2}\par
{\f1{}\pard\li720{}ISO646-NO2}, {\f1{}iso-ir-61} and {\f1{}no2} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}NeXTSTEP}\par
{\f1{}\pard\li720{}next} is an alias for this charset.  Source: Peter Svanberg - psv@nada.kth.se.\par
\par
\pard{}{\f1{}PT}\par
{\f1{}\pard\li720{}ISO646-PT} and {\f1{}iso-ir-16} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}PT2}\par
{\f1{}\pard\li720{}ISO646-PT2} and {\f1{}iso-ir-84} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}SEN_850200_B}\par
{\f1{}\pard\li720{}FI}, {\f1{}ISO646-FI}, {\f1{}ISO646-SE}, {\f1{}SS636127}, {\f1{}iso-ir-10} and {\f1{}se} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}SEN_850200_C}\par
{\f1{}\pard\li720{}ISO646-SE2}, {\f1{}iso-ir-11} and {\f1{}se2} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}T.61-7bit}\par
{\f1{}\pard\li720{}iso-ir-102} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}baltic}\par
{\f1{}\pard\li720{}iso-ir-179} is an alias for this charset.  Source: ISO 2375 registry.  &g1esc x2d56 &g2esc x2e56 &g3esc x2f56.\par
\par
\pard{}{\f1{}greek-ccitt}\par
{\f1{}\pard\li720{}iso-ir-150} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}greek7}\par
{\f1{}\pard\li720{}iso-ir-88} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}greek7-old}\par
{\f1{}\pard\li720{}iso-ir-18} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}hp-roman8}\par
{\f1{}\pard\li720{}r8} and {\f1{}roman8} are aliases for this charset.  Source: LaserJet IIP Printer User's Manual,.  HP part no 33471-90901, Hewlet-Packard, June 1989.\par
\par
\pard{}{\f1{}latin-greek}\par
{\f1{}\pard\li720{}iso-ir-19} is an alias for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}{\f1{}mac-is}\par

\pard{}{\f1{}macintosh}\par
{\f1{}\pard\li720{}mac} is an alias for this charset.  Source: The Unicode Standard ver 1.0, ISBN 0-201-56788-1, Oct 1991.\par
\par
\pard{}{\f1{}macintosh_ce}\par
{\f1{}\pard\li720{}macce} is an alias for this charset.  Source: Macintosh CE fonts.\par
\par
\pard{}{\f1{}sami}\par
{\f1{}\pard\li720{}iso-ir-158}, {\f1{}lap} and {\f1{}latin-lap} are aliases for this charset.  Source: ISO 2375 registry.\par
\par
\pard{}}\par
\page\pard Node: {\b ASCII misc}, \keepn Next: {\uldb IBM and MS}{\v IBM_and_MS}, Prev: {\uldb Tabular}{\v Tabular}, Up: {\uldb Top}{\v Top}\line
K{\footnote K ASCII misc}
#{\footnote ASCII_misc}
${\footnote \pard{}ASCII and some derivatives}\par
\pard{\fs24\b ASCII and some derivatives}\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb ASCII}{\v ASCII}\tab Usual ASCII\par
{\uldb ISO 8859}{\v ISO_8859}\tab ASCII extended by Latin Alphabets\par
{\uldb ASCII-BS}{\v ASCII_BS}\tab ASCII 7-bits, BS to overstrike\par
{\uldb flat}{\v flat}\tab ASCII without diacritics nor underline\par
\pard{}}\par
\page\pard Node: {\b ASCII}, \keepn Next: {\uldb ISO 8859}{\v ISO_8859}, Prev: {\uldb ASCII misc}{\v ASCII_misc}, Up: {\uldb ASCII misc}{\v ASCII_misc}\line
K{\footnote K ASCII}
#{\footnote ASCII}
${\footnote \pard{}Usual ASCII}\par
\pard{\fs24\b Usual ASCII}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}ASCII}.  In fact, it's true name is {\f1{}ANSI_X3.4-1968} as per RFC 1345, accepted aliases being {\f1{}ANSI_X3.4-1986}, {\f1{}ASCII}, {\f1{}IBM367}, {\f1{}ISO646-US}, {\f1{}ISO_646.irv:1991}, {\f1{}US-ASCII}, {\f1{}cp367}, {\f1{}iso-ir-6} and {\f1{}us}.  The shortest way of specifying it in {\f1{}recode} is {\f1{}us}.\par
\par
\pard{}This documentation used to include ASCII tables.  They have been removed since the {\f1{}recode} program can now recreate these easily:\par
\par
{\pard\keep\li720\f1{}recode -lf us                   for commented ASCII\line
recode -ld us                   for concise decimal table\line
recode -lo us                   for concise octal table\line
recode -lh us                   for concise hexadecimal table\par
\pard\f0{}}\par
\page\pard Node: {\b ISO 8859}, \keepn Next: {\uldb ASCII-BS}{\v ASCII_BS}, Prev: {\uldb ASCII}{\v ASCII}, Up: {\uldb ASCII misc}{\v ASCII_misc}\line
K{\footnote K ISO 8859}
#{\footnote ISO_8859}
${\footnote \pard{}ASCII extended by Latin Alphabets}\par
\pard{\fs24\b ASCII extended by Latin Alphabets}\par
\par
\pard{}There are many Latin charsets.  The following has been written by Tim Lasko <lasko@video.dec.com>, a long while ago:\par
\par
{\pard\li720{}ISO Latin-1, or more completely ISO Latin Alphabet No 1, is now an international standard as of February 1987 (IS 8859, Part 1).  For those American USEnet'rs that care, the 8-bit ASCII standard, which is essentially the same code, is going through the final administrative processes prior to publication.  ISO Latin-1 (IS 8859/1) is actually one of an entire family of eight-bit one-byte character sets, all having ASCII on the left hand side, and with varying repertoires on the right hand side:\par
\par
{\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin Alphabet No 1 (caters to Western Europe - now approved).\par
\par
\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin Alphabet No 2 (caters to Eastern Europe - now approved).\par
\par
\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin Alphabet No 3 (caters to SE Europe + others - in draft ballot).\par
\par
\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin Alphabet No 4 (caters to Northern Europe - in draft ballot).\par
\par
\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin-Cyrillic alphabet (right half all Cyrillic - processing currently suspended pending USSR input).\par
\par
\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin-Arabic alphabet (right half all Arabic - now approved).\par
\par
\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin-Greek alphabet (right half Greek + symbols - in draft ballot).\par
\par
\pard\fi-720\li1440\tx864\tx1440{}\tab{}{\f2\'B7}\tab{}Latin-Hebrew alphabet (right half Hebrew + symbols - proposed).\par
\pard\li720{}}\par
\pard{}}\par
\pard{}The ISO Latin Alphabet 1 is available as a charset in {\f1{}recode} under the name {\f1{}Latin-1}.  In fact, it's true name is {\f1{}ISO_8859-1:1987} as per RFC 1345, accepted aliases being {\f1{}CP819}, {\f1{}IBM819}, {\f1{}ISO-8859-1}, {\f1{}ISO_8859-1}, {\f1{}iso-ir-100}, {\f1{}l1} and {\f1{}Latin-1}.  The shortest way of specifying it in {\f1{}recode} is {\f1{}l1}.\par
\par
\pard{}It is an eight-bit code which coincides with ASCII for the lower half.  This documentation used to include Latin-1 tables.  They have been removed since the {\f1{}recode} program can now recreate these easily:\par
\par
{\pard\keep\li720\f1{}recode -lf l1                   for commented ISO Latin-1\line
recode -ld l1                   for concise decimal table\line
recode -lo l1                   for concise octal table\line
recode -lh l1                   for concise hexadecimal table\par
\pard\f0{}}\par
\page\pard Node: {\b ASCII-BS}, \keepn Next: {\uldb flat}{\v flat}, Prev: {\uldb ISO 8859}{\v ISO_8859}, Up: {\uldb ASCII misc}{\v ASCII_misc}\line
K{\footnote K ASCII-BS}
#{\footnote ASCII_BS}
${\footnote \pard{}ASCII 7-bits, {\f1{}BS} to overstrike}\par
\pard{\fs24\b ASCII 7-bits, {\f1{}BS} to overstrike}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}ASCII-BS}, with {\f1{}BS} as an acceptable alias.\par
\par
\pard{}The file is straight ASCII, seven bits only.  According to the definition of ASCII, diacritics are applied by a sequence of three characters: the letter, one {\f1{}BS}, the diacritic mark.  We deviate slightly from this by exchanging the diacritic mark and the letter so, on a screen device, the diacritic will disappear and let the letter alone.  At recognition time, both methods are acceptable.\par
\par
\pard{}The French quotes are coded by the sequences: {\f1{}< BS "} or {\f1{}" BS <} for the opening quote and {\f1{}> BS "} or {\f1{}" BS >} for the closing quote.  This artificial convention was inherited in straight {\f1{}ASCII-BS} from habits around {\f1{}Bang-Bang} entry, and is not well known.  But we decided to stick to it so that {\f1{}ASCII-BS} charset will not lose French quotes.\par
\par
\pard{}The {\f1{}ASCII-BS} charset is independent of {\f1{}ASCII}, and different.  The following examples demonstrate this, knowing at advance that {\f1{}!2} is the {\f1{}Bang-Bang} way of representing an {\f1{}e} with an acute accent.  Compare:\par
\par
{\pard\keep\li720\f1{}% echo \'5C!2 | recode -v bang..l1/d\line
Request: Bang-Bang..ISO-8859-1/Decimal-1\line
233,  10\par
\pard\f0{}}\par
\pard{}with:\par
\par
{\pard\keep\li720\f1{}% echo \'5C!2 | recode -v bang..bs/d\line
Request: Bang-Bang..ISO-8859-1..ASCII-BS/Decimal-1\line
 39,   8, 101,  10\par
\pard\f0{}}\par
\pard{}In the first case, the {\f1{}e} with an acute accent is merely transmitted by the {\f1{}Latin-1..ASCII} mapping, not having a special recoding rule for it.  In the {\f1{}Latin-1..ASCII-BS} case, the acute accent is applied over the {\f1{}e} with a backspace: diacriticised characters have special rules.  For the {\f1{}ASCII-BS} charset, reversibility is still possible, but there might be difficult cases.\par
\par
\page\pard Node: {\b flat}, \keepn Next: {\uldb }{\v }, Prev: {\uldb ASCII-BS}{\v ASCII_BS}, Up: {\uldb ASCII misc}{\v ASCII_misc}\line
K{\footnote K flat}
#{\footnote flat}
${\footnote \pard{}ASCII without diacritics nor underline}\par
\pard{\fs24\b ASCII without diacritics nor underline}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}flat}.\par
\par
\pard{}This code is ASCII expunged of all diacritics and underlines, as long as they are applied using three character sequences, with {\f1{}BS} in the middle.  Also, despite slightly unrelated, each control character is represented by a sequence of two or three graphic characters.  The newline character, however, keeps its functionality and is not represented.\par
\par
\pard{}Note that charset {\f1{}flat} is a terminal charset.  We can convert {\i to} {\f1{}flat}, but not {\i from} it.\par
\par
\page\pard Node: {\b IBM and MS}, \keepn Next: {\uldb CDC}{\v CDC}, Prev: {\uldb ASCII misc}{\v ASCII_misc}, Up: {\uldb Top}{\v Top}\line
K{\footnote K IBM and MS}
#{\footnote IBM_and_MS}
${\footnote \pard{}Some IBM or Microsoft charsets}\par
\pard{\fs24\b Some IBM or Microsoft charsets}\par
\par
\pard{}The {\f1{}recode} program provides various IBM or Microsoft code pages (see {\uldb Tabular}{\v Tabular}).  An easy way to find them all at once out of the {\f1{}recode} program itself is through the command:\par
\par
{\pard\keep\li720\f1{}recode -l | egrep -i '(CP|IBM)[0-9]'\par
\pard\f0{}}\par
\pard{}But also, see few special charsets presented in the incoming sections.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb EBCDIC}{\v EBCDIC}\tab EBCDIC codes\par
{\uldb IBM-PC}{\v IBM_PC}\tab IBM's PC code\par
{\uldb Icon-QNX}{\v Icon_QNX}\tab Unisys' Icon code\par
\pard{}}\par
\page\pard Node: {\b EBCDIC}, \keepn Next: {\uldb IBM-PC}{\v IBM_PC}, Prev: {\uldb IBM and MS}{\v IBM_and_MS}, Up: {\uldb IBM and MS}{\v IBM_and_MS}\line
K{\footnote K EBCDIC}
#{\footnote EBCDIC}
${\footnote \pard{}EBCDIC code}\par
\pard{\fs24\b EBCDIC code}\par
\par
\pard{}This charset is the IBM's External Binary Coded Decimal for Interchange Coding.  This is an eight bits code.  The following three variants were implemented in {\f1{}recode} independently of RFC 1345:\par
\par
{\pard{}{\f1{}EBCDIC}\par
\pard\li720{}In {\f1{}recode}, the {\f1{}us..ebcdic} conversion is identical to {\f1{}dd conv=ebcdic} conversion, and {\f1{}recode} {\f1{}ebcdic..us} conversion is identical to {\f1{}dd conv=ascii} conversion.  This charset also represents the way Control Data Corporation relates EBCDIC to 8-bits ASCII.\par
\par
\pard{}{\f1{}EBCDIC-CCC}\par
\pard\li720{}In {\f1{}recode}, the {\f1{}us..ebcdic-ccc} or {\f1{}ebcdic-ccc..us} conversions represent the way Concurrent Computer Corporation (formerly Perkin Elmer) relates EBCDIC to 8-bits ASCII.\par
\par
\pard{}{\f1{}EBCDIC-IBM}\par
\pard\li720{}In {\f1{}recode}, the {\f1{}us..ebcdic-ibm} conversion is {\i almost} identical to the GNU {\f1{}dd conv=ibm} conversion.  Given the exact {\f1{}dd conv=ibm} conversion table, {\f1{}recode} once said:\par
\par
{\pard\keep\li1440\f1{}Codes  91 and 213 both recode to 173\line
Codes  93 and 229 both recode to 189\line
No character recodes to  74\line
No character recodes to 106\par
\pard\li720\f0{}}\par
\pard\li720{}So I arbitrarily chose to recode 213 by 74 and 229 by 106.  This makes the {\f1{}EBCDIC-IBM} recoding reversible, but this is not necessarily the best correction.  In any case, I think that GNU {\f1{}dd} should be amended.  {\f1{}dd} and {\f1{}recode} should ideally agree on the same correction.  So, this table might change once again.\par
\par
\pard{}}\par
\pard{}RFC 1345 brings into {\f1{}recode} 15 other EBCDIC charsets, and 21 other charsets having EBCDIC in at least one of their alias names.  You can get a list of all these by executing:\par
\par
{\pard\keep\li720\f1{}recode -l | grep -i ebcdic\par
\pard\f0{}}\par
\pard{}Note that {\f1{}recode} may convert a pure stream of EBCDIC characters, but it does not know how to handle binary data between records which is sometimes used to delimit them and build physical blocks.  If end of lines are not marked, fixed record size may produce something readable, but {\f1{}VB} or {\f1{}VBS} blocking is likely to yield some garbage in the converted results.\par
\par
\page\pard Node: {\b IBM-PC}, \keepn Next: {\uldb Icon-QNX}{\v Icon_QNX}, Prev: {\uldb EBCDIC}{\v EBCDIC}, Up: {\uldb IBM and MS}{\v IBM_and_MS}\line
K{\footnote K IBM-PC}
#{\footnote IBM_PC}
${\footnote \pard{}IBM's PC code}\par
\pard{\fs24\b IBM's PC code}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}IBM-PC}, with {\f1{}dos}, {\f1{}MSDOS} and {\f1{}pc} as acceptable aliases.  The shortest way of specifying it in {\f1{}recode} is {\f1{}pc}.\par
\par
\pard{}The charset is aimed towards a PC microcomputer from IBM or any compatible.  This is an eight-bit code.  This charset is fairly old in {\f1{}recode}, its tables were produced a long while ago by mere inspection of a printed chart of the IBM-PC codes and glyph.\par
\par
\pard{}It has {\f1{}CR-LF} as its implied surface.  This means that, if the original end of lines have to be preserved while going out of {\f1{}IBM-PC}, they should currently be added back through the usage of a surface on the other charset, or better, just never removed.  Here are examples for both cases:\par
\par
{\pard\keep\li720\f1{}recode pc..l2/cl < {\i input} > {\i output}\line
recode pc/..l2 < {\i input} > {\i output}\par
\pard\f0{}}\par
\pard{}RFC 1345 brings into {\f1{}recode} 44 {\f1{}IBM} charsets or code pages, and also 8 other code pages.  You can get a list of these all these by executing:({\ul 1}{\v IBM_PC_1})\par
\par
{\pard\keep\li720\f1{}recode -l | egrep -i '(CP|IBM)[0-9]'\par
\pard\f0{}}\par
\pard{}All charset or aliases beginning with letters {\f1{}CP} or {\f1{}IBM} also have {\f1{}CR-LF} as their implied surface.  The same is true for a purely numeric alias in the same family.  For example, all of {\f1{}819}, {\f1{}CP819} and {\f1{}IBM819} imply {\f1{}CR-LF} as a surface.  Note that {\f1{}ISO-8859-1} does {\i not} imply a surface, despite it shares the same tabular data as {\f1{}819}.\par
\par
\pard{}There are a few discrepancies between this {\f1{}IBM-PC} charset and the very similar RFC 1345 charset {\f1{}ibm437}, which have not been analysed yet, so the charsets are being kept separate for now.  This might change in the future, and the {\f1{}IBM-PC} charset might disappear.  Wizards would be interested in comparing the output of these two commands:\par
\par
{\pard\keep\li720\f1{}recode -vh IBM-PC..Latin-1\line
recode -vh IBM437..Latin-1\par
\pard\f0{}}\par
\pard{}The first command uses the charset prior to RFC 1345 introduction.  Both methods give different recodings.  These differences are annoying, the fuzziness will have to be explained and settle down one day.\par
\par
\page\pard#{\footnote IBM_PC_1}
\pard{}(1) On DOS/Windows, stock shells do not know that apostrohes quote special characters like {\f1{}|}, so one need to use double quotes instead of apostrophes.\par
\par
\page\pard Node: {\b Icon-QNX}, \keepn Next: {\uldb }{\v }, Prev: {\uldb IBM-PC}{\v IBM_PC}, Up: {\uldb IBM and MS}{\v IBM_and_MS}\line
K{\footnote K Icon-QNX}
#{\footnote Icon_QNX}
${\footnote \pard{}Unisys' Icon code}\par
\pard{\fs24\b Unisys' Icon code}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}Icon-QNX}, with {\f1{}QNX} as an acceptable alias.\par
\par
\pard{}The file is using Unisys' Icon way to represent diacritics with code 25 escape sequences, under the system QNX.  This is a seven-bit code, even if eight-bit codes can flow through as part of IBM-PC charset.\par
\par
\page\pard Node: {\b CDC}, \keepn Next: {\uldb Micros}{\v Micros}, Prev: {\uldb IBM and MS}{\v IBM_and_MS}, Up: {\uldb Top}{\v Top}\line
K{\footnote K CDC}
#{\footnote CDC}
${\footnote \pard{}Charsets for CDC machines}\par
\pard{\fs24\b Charsets for CDC machines}\par
\par
\pard{}What is now {\f1{}recode} evolved out, through many transformations really, from a set of programs which were originally written in Control Data Corporation's assembler ({\f1{}COMPASS}), with bits in FORTRAN, and later, rewritten in CDC 6000 Pascal.  The CDC heritage shows by the fact some old CDC charsets are still supported.\par
\par
\pard{}The {\f1{}recode} author used to be familiar with CDC Scope-NOS/BE and Kronos-NOS, and many CDC formats.  Reading CDC tapes directly on other machines is often a challenge, and {\f1{}recode} does not always solve it.  It helps having tapes created in coded mode instead of binary mode, and using {\f1{}S} (Stranger) tapes instead of {\f1{}I} (Internal) tapes.  ANSI labels and multi-file tapes might be the source of trouble.  There are ways to handle a few Cyber Record Manager formats, but some of them might be quite difficult to decode properly after the transfer is done.\par
\par
\pard{}The {\f1{}recode} program is usable only for a small subset of NOS text formats, and surely not with binary textual formats, like {\f1{}UPDATE} or {\f1{}MODIFY} sources, for example.  {\f1{}recode} is not especially suited for reading 8/12 or 56/60 packing, yet this could easily arranged if there was a demand for it.  It does not have the ability to translate Display Code directly, as the ASCII conversion implied by tape drivers or FTP does the initial approximation.  {\f1{}recode} can decode 6/12 caret notation over Display Code already mapped to ASCII.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Display Code}{\v Display_Code}\tab Control Data's Display Code\par
{\uldb CDC-NOS}{\v CDC_NOS}\tab ASCII 6/12 from NOS\par
{\uldb Bang-Bang}{\v Bang_Bang}\tab ASCII ``bang bang''\par
\pard{}}\par
\page\pard Node: {\b Display Code}, \keepn Next: {\uldb CDC-NOS}{\v CDC_NOS}, Prev: {\uldb CDC}{\v CDC}, Up: {\uldb CDC}{\v CDC}\line
K{\footnote K Display Code}
#{\footnote Display_Code}
${\footnote \pard{}Control Data's Display Code}\par
\pard{\fs24\b Control Data's Display Code}\par
\par
\pard{}This code is not available in {\f1{}recode}, but repeated here for reference.  This is a 6-bit code used on CDC mainframes.\par
\par
{\pard\keep\li720\f1{}Octal display code to graphic       Octal display code to octal ASCII\line
\line
00  :    20  P    40  5   60  #     00 072  20 120  40 065  60 043\line
01  A    21  Q    41  6   61  [     01 101  21 121  41 066  61 133\line
02  B    22  R    42  7   62  ]     02 102  22 122  42 067  62 135\line
03  C    23  S    43  8   63  %     03 103  23 123  43 070  63 045\line
04  D    24  T    44  9   64  "     04 104  24 124  44 071  64 042\line
05  E    25  U    45  +   65  _     05 105  25 125  45 053  65 137\line
06  F    26  V    46  -   66  !     06 106  26 126  46 055  66 041\line
07  G    27  W    47  *   67  &     07 107  27 127  47 052  67 046\line
10  H    30  X    50  /   70  '     10 110  30 130  50 057  70 047\line
11  I    31  Y    51  (   71  ?     11 111  31 131  51 050  71 077\line
12  J    32  Z    52  )   72  <     12 112  32 132  52 051  72 074\line
13  K    33  0    53  $   73  >     13 113  33 060  53 044  73 076\line
14  L    34  1    54  =   74  @     14 114  34 061  54 075  74 100\line
15  M    35  2    55      75  \'5C     15 115  35 062  55 040  75 134\line
16  N    36  3    56  ,   76  ^     16 116  36 063  56 054  76 136\line
17  O    37  4    57  .   77  ;     17 117  37 064  57 056  77 073\par
\pard\f0{}}\par
\pard{}In older times, {\f1{}:} used octal 63, and octal 0 was not a character.  The table above shows the ASCII glyph interpretation of codes 60 to 77, yet these 16 codes were once defined differently.\par
\par
\pard{}There is no explicit end of line in Display Code, and the Cyber Record Manager introduced many new ways to represent them, the traditional end of lines being reachable by setting {\f1{}RT} to {\f1{}Z}.  If 6-bit bytes in a file are sequentially counted from 1, a traditional end of line does exist if bytes 10*{\i n}+9 and 10{\i n}+10 are both zero for a given {\i n}, in which case these two bytes are not to be interpreted as {\f1{}::}.  Also, up to 9 immediately preceeding zero bytes, going backward, are to be considered as part of the end of line and not interpreted as {\f1{}:}({\ul 1}{\v Display_Code_1}).\par
\par
\page\pard#{\footnote Display_Code_1}
\pard{}(1) This convention replaced an older one saying that up to 4 immediately preceeding {\i pairs} of zero bytes, going backward, are to be considered as part of the end of line and not interpreted as {\f1{}::}.\par
\par
\page\pard Node: {\b CDC-NOS}, \keepn Next: {\uldb Bang-Bang}{\v Bang_Bang}, Prev: {\uldb Display Code}{\v Display_Code}, Up: {\uldb CDC}{\v CDC}\line
K{\footnote K CDC-NOS}
#{\footnote CDC_NOS}
${\footnote \pard{}ASCII 6/12 from NOS}\par
\pard{\fs24\b ASCII 6/12 from NOS}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}CDC-NOS}, with {\f1{}NOS} as an acceptable alias.\par
\par
\pard{}This is one of the charset in use on CDC Cyber NOS systems to represent ASCII, sometimes named \'A2NOS 6/12\'A2 code for coding ASCII.  This code is also known as \'A2caret ASCII\'A2.  It is based on a six bits character set in which small letters and control characters are coded using a {\f1{}^} escape and, sometimes, a {\f1{}@} escape.\par
\par
\pard{}The routines given here presume that the six bits code is already expressed in ASCII by the communication channel, with embedded ASCII {\f1{}^} and {\f1{}@} escapes.\par
\par
\pard{}Here is a table showing which characters are being used to encode each ASCII character.\par
\par
{\pard\keep\li720\f1{}000  ^5  020  ^#  040     060  0  100 @A  120  P  140  @G  160  ^P\line
001  ^6  021  ^[  041  !  061  1  101  A  121  Q  141  ^A  161  ^Q\line
002  ^7  022  ^]  042  "  062  2  102  B  122  R  142  ^B  162  ^R\line
003  ^8  023  ^%  043  #  063  3  103  C  123  S  143  ^C  163  ^S\line
004  ^9  024  ^"  044  $  064  4  104  D  124  T  144  ^D  164  ^T\line
005  ^+  025  ^_  045  %  065  5  105  E  125  U  145  ^E  165  ^U\line
006  ^-  026  ^!  046  &  066  6  106  F  126  V  146  ^F  166  ^V\line
007  ^*  027  ^&  047  '  067  7  107  G  127  W  147  ^G  167  ^W\line
010  ^/  030  ^'  050  (  070  8  110  H  130  X  150  ^H  170  ^X\line
011  ^(  031  ^?  051  )  071  9  111  I  131  Y  151  ^I  171  ^Y\line
012  ^)  032  ^<  052  *  072 @D  112  J  132  Z  152  ^J  172  ^Z\line
013  ^$  033  ^>  053  +  073  ;  113  K  133  [  153  ^K  173  ^0\line
014  ^=  034  ^@  054  ,  074  <  114  L  134  \'5C  154  ^L  174  ^1\line
015  ^   035  ^\'5C  055  -  075  =  115  M  135  ]  155  ^M  175  ^2\line
016  ^,  036  ^^  056  .  076  >  116  N  136 @B  156  ^N  176  ^3\line
017  ^.  037  ^;  057  /  077  ?  117  O  137  _  157  ^O  177  ^4\par
\pard\f0{}}\par
\page\pard Node: {\b Bang-Bang}, \keepn Next: {\uldb }{\v }, Prev: {\uldb CDC-NOS}{\v CDC_NOS}, Up: {\uldb CDC}{\v CDC}\line
K{\footnote K Bang-Bang}
#{\footnote Bang_Bang}
${\footnote \pard{}ASCII "bang bang"}\par
\pard{\fs24\b ASCII "bang bang"}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}Bang-Bang}.\par
\par
\pard{}This code, in use on Cybers at Universite' de Montre'al mainly, served to code a lot of French texts.  The original name of this charset is \'A2ASCII code' Display\'A2.  This code is also known as \'A2Bang-bang\'A2.  It is based on a six bits character set in which capitals, French diacritics and a few others are coded using an {\f1{}!} escape followed by a single character, and control characters using a double {\f1{}!} escape followed by a single character.\par
\par
\pard{}The routines given here presume that the six bits code is already expressed in ASCII by the communication channel, with embedded ASCII {\f1{}!} escapes.\par
\par
\pard{}Here is a table showing which characters are being used to encode each ASCII character.\par
\par
{\pard\keep\li720\f1{}000 !!@  020 !!P  040    060 0  100 @   120 !P  140 !@ 160 P\line
001 !!A  021 !!Q  041 !" 061 1  101 !A  121 !Q  141 A  161 Q\line
002 !!B  022 !!R  042 "  062 2  102 !B  122 !R  142 B  162 R\line
003 !!C  023 !!S  043 #  063 3  103 !C  123 !S  143 C  163 S\line
004 !!D  024 !!T  044 $  064 4  104 !D  124 !T  144 D  164 T\line
005 !!E  025 !!U  045 %  065 5  105 !E  125 !U  145 E  165 U\line
006 !!F  026 !!V  046 &  066 6  106 !F  126 !V  146 F  166 V\line
007 !!G  027 !!W  047 '  067 7  107 !G  127 !W  147 G  167 W\line
010 !!H  030 !!X  050 (  070 8  110 !H  130 !X  150 H  170 X\line
011 !!I  031 !!Y  051 )  071 9  111 !I  131 !Y  151 I  171 Y\line
012 !!J  032 !!Z  052 *  072 :  112 !J  132 !Z  152 J  172 Z\line
013 !!K  033 !![  053 +  073 ;  113 !K  133 [   153 K  173 ![\line
014 !!L  034 !!\'5C  054 ,  074 <  114 !L  134 \'5C   154 L  174 !\'5C\line
015 !!M  035 !!]  055 -  075 =  115 !M  135 ]   155 M  175 !]\line
016 !!N  036 !!^  056 .  076 >  116 !N  136 ^   156 N  176 !^\line
017 !!O  037 !!_  057 /  077 ?  117 !O  137 _   157 O  177 !_\par
\pard\f0{}}\par
\page\pard Node: {\b Micros}, \keepn Next: {\uldb Miscellaneous}{\v Miscellaneous}, Prev: {\uldb CDC}{\v CDC}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Micros}
#{\footnote Micros}
${\footnote \pard{}Other micro-computer charsets}\par
\pard{\fs24\b Other micro-computer charsets}\par
\par
\pard{}The {\f1{}NeXT} charset, which use to be especially provided in releases of {\f1{}recode} before 3.5, has been integrated since as one RFC 1345 table.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Apple-Mac}{\v Apple_Mac}\tab Apple's Macintosh code\par
{\uldb AtariST}{\v AtariST}\tab Atari ST code\par
\pard{}}\par
\page\pard Node: {\b Apple-Mac}, \keepn Next: {\uldb AtariST}{\v AtariST}, Prev: {\uldb Micros}{\v Micros}, Up: {\uldb Micros}{\v Micros}\line
K{\footnote K Apple-Mac}
#{\footnote Apple_Mac}
${\footnote \pard{}Apple's Macintosh code}\par
\pard{\fs24\b Apple's Macintosh code}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}Apple-Mac}.  The shortest way of specifying it in {\f1{}recode} is {\f1{}ap}.\par
\par
\pard{}The charset is aimed towards a Macintosh micro-computer from Apple.  This is an eight bit code.  The file is the data fork only.  This charset is fairly old in {\f1{}recode}, its tables were produced a long while ago by mere inspection of a printed chart of the Macintosh codes and glyph.\par
\par
\pard{}It has {\f1{}LF} as its implied surface.  This means that, if the original end of lines have to be preserved while going out of {\f1{}Apple-Mac}, they should currently be added back through the usage of a surface on the other charset, or better, just never removed.  Here are examples for both cases:\par
\par
{\pard\keep\li720\f1{}recode ap..l2/cl < {\i input} > {\i output}\line
recode ap/..l2 < {\i input} > {\i output}\par
\pard\f0{}}\par
\pard{}RFC 1345 brings into {\f1{}recode} 2 other Macintosh charsets.  You can discover them by using {\f1{}grep} over the output of {\f1{}recode -l}:\par
\par
{\pard\keep\li720\f1{}recode -l | grep -i mac\par
\pard\f0{}}\par
\pard{}Charsets {\f1{}macintosh} and {\f1{}macintosh_ce}, as well as their aliases {\f1{}mac} and {\f1{}macce} also have {\f1{}CR} as their implied surface.\par
\par
\pard{}There are a few discrepancies between the {\f1{}Apple-Mac} charset and the very similar RFC 1345 charset {\f1{}macintosh}, which have not been analysed yet, so the charsets are being kept separate for now.  This might change in the future, and the {\f1{}Apple-Mac} charset might disappear.  Wizards would be interested in comparing the output of these two commands:\par
\par
{\pard\keep\li720\f1{}recode -vh Apple-Mac..Latin-1\line
recode -vh macintosh..Latin-1\par
\pard\f0{}}\par
\pard{}The first command use the charset prior to RFC 1345 introduction.  Both methods give different recodings.  These differences are annoying, the fuzziness will have to be explained and settle down one day.\par
\par
\pard{}As a side note, some people ask if there is a Macintosh port of the {\f1{}recode} program.  I'm not aware of any.  I presume that if the tool fills a need for Macintosh users, someone will port it one of these days?\par
\par
\page\pard Node: {\b AtariST}, \keepn Next: {\uldb }{\v }, Prev: {\uldb Apple-Mac}{\v Apple_Mac}, Up: {\uldb Micros}{\v Micros}\line
K{\footnote K AtariST}
#{\footnote AtariST}
${\footnote \pard{}Atari ST code}\par
\pard{\fs24\b Atari ST code}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}AtariST}.\par
\par
\pard{}This is the character set used on the Atari ST/TT/Falcon.  This is similar to {\f1{}IBM-PC}, but differs in some details: it includes some more accented characters, the graphic characters are mostly replaced by Hebrew characters, and there is a true German {\f1{}sharp s} different from Greek {\f1{}beta}.\par
\par
\pard{}About the end-of-line conversions: the canonical end-of-line on the Atari is {\f1{}\'5Cr\'5Cn}, but unlike {\f1{}IBM-PC}, the OS makes no difference between text and binary input/output; it is up to the application how to interpret the data.  In fact, most of the libraries that come with compilers can grok both {\f1{}\'5Cr\'5Cn} and {\f1{}\'5Cn} as end of lines.  Many of the users who also have access to Unix systems prefer {\f1{}\'5Cn} to ease porting Unix utilities.  So, for easing reversibility, {\f1{}recode} tries to let {\f1{}\'5Cr} undisturbed through recodings.\par
\par
\page\pard Node: {\b Miscellaneous}, \keepn Next: {\uldb Surfaces}{\v Surfaces}, Prev: {\uldb Micros}{\v Micros}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Miscellaneous}
#{\footnote Miscellaneous}
${\footnote \pard{}Various other charsets}\par
\pard{\fs24\b Various other charsets}\par
\par
\pard{}Even if these charsets were originally added to {\f1{}recode} for handling texts written in French, they find other uses.  We did use them lot for writing French diacriticised texts in the past, so {\f1{}recode} knows how to handle these particularly well for French texts.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb HTML}{\v HTML}\tab World Wide Web representations\par
{\uldb LaTeX}{\v LaTeX}\tab LaTeX macro calls\par
{\uldb Texinfo}{\v Texinfo}\tab GNU project documentation files\par
{\uldb African}{\v African}\tab African charsets\par
{\uldb Cyrillic}{\v Cyrillic}\tab Cyrillic charsets\par
{\uldb Texte}{\v Texte}\tab Easy French conventions\par
{\uldb Mule}{\v Mule}\tab Mule as a multiplexed charset\par
\pard{}}\par
\page\pard Node: {\b HTML}, \keepn Next: {\uldb LaTeX}{\v LaTeX}, Prev: {\uldb Miscellaneous}{\v Miscellaneous}, Up: {\uldb Miscellaneous}{\v Miscellaneous}\line
K{\footnote K HTML}
#{\footnote HTML}
${\footnote \pard{}World Wide Web representations}\par
\pard{\fs24\b World Wide Web representations}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}HTML}, with {\f1{}h4} as an acceptable alias.\par
\par
\pard{}HTML texts used by World Wide Web often use special sequences, beginning with an ampersand {\f1{}&} and ending with a semicolon {\f1{};}, for representing characters.  The sequence may itself start with a number sigh {\f1{}#} and be followed by digits, so forming a \'A2numeric character reference\'A2, or else be an alphabetic identifier, so forming a \'A2character entity reference\'A2.\par
\par
\pard{}Printable characters from Latin-1 may be used directly in an HTML text.  However, partly because people have deficient keyboards, partly because people want to transmit HTML texts over non 8-bit clean channels while not using MIME, it is common (yet debatable) to use character entity references even for Latin-1 characters, when they fall outside ASCII (that is, when they have the 8th bit set).\par
\par
\pard{}When you recode from another charset to {\f1{}HTML}, beware that all occurrences of double quotes, ampersands, and left or right angle brackets are translated into special sequences.  However, in practice, people often use ampersands and angle brackets in the other charset for introducing HTML commands, compromising it: it is not pure HTML, not it is pure other charset.  These particular translations can be rather inconvenient, they may be specifically inhibited through the command option {\f1{}-d} (see {\uldb Mixed}{\v Mixed}).\par
\par
\pard{}Codes not having a mnemonic entity are output by {\f1{}recode} using the {\f1{}&#{\i nnn};} notation, where {\i nnn} is a decimal representation of the UCS code value.  When there is an entity name for a character, it is always preferred over a numeric character reference.  ASCII printable characters are always generated directly.  So is the newline.  While reading HTML, {\f1{}recode} supports numeric character reference as alternate writings, even when written as hexadecimal numbers, as in {\f1{}&#xfffd}.  This is documented in:\par
\par
{\pard\keep\li720\f1{}http://www.w3.org/TR/REC-html40/intro/sgmltut.html#h-3.2.3\par
\pard\f0{}}\par
\pard{}About levels of HTML, Fran\'E7ois Yergeau <yergeau@alis.com> writes:\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}HTML 1.0 was never really documented.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}HTML 2.0 entities are listed in RFC 1866.  Basically, there is an entity for each {\i alphabetical} character in the right part of ISO 8859-1.  In addition, there are four entities for syntax-significant ASCII characters: {\f1{}&amp;}, {\f1{}&gt;}, {\f1{}&lt;} and {\f1{}&quot;}.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}RFC 2070 (HTML-i18n) added entities to cover the whole right part of ISO 8859-1.  The list is conveniently accessible at <URL:http://www.alis.com:8085/ietf/html/html-latin1.sgml>.  In addition, four i18n-related entities were added: {\f1{}&zwnj;} ({\f1{}&#8204;}), {\f1{}&zwj;} ({\f1{}&#8205;}), {\f1{}&lrm;} ({\f1{}&#8206}) and {\f1{}&rlm;} ({\f1{}&#8207;}).\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}HTML 3.2 (<URL:http://www.w3.org/TR/REC-html32.html>) took up the full Latin-1 list but not the i18n-related entities from RFC 2070.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}HTML 4.0 (<URL:http://www.w3.org/TR/REC-html40/>) has the whole Latin-1 list, a set of entities for symbols, mathematical symbols, and Greek letters, and another set for markup-significant and internationalization characters comprising the 4 ASCII entities, the 4 i18n-related from RFC 2070 plus some more.\par
\par
\pard{}}\par
\pard{}When {\f1{}recode} translates to HTML, the translation occurs according to <URL:http://www.w3.org/TR/REC-html40/sgml/entities.html>.  It is also assumed that RFC 1866 has an equivalent contents.  When translating {\i from} HTML, {\f1{}recode} accepts some alternative special sequences, to be forgiving when files use older HTML tables.\par
\par
\pard{}The {\f1{}recode} program can be used to {\i normalise} an HTML file using oldish conventions.  For example, it accepts {\f1{}&AE;}, as this once was a valid writing, somewhere.  However, it should always produce {\f1{}&AElig;} instead of {\f1{}&AE;}.  Yet, this is not completely true.  If one does:\par
\par
{\pard\keep\li720\f1{}recode h3..h3 < {\i input}\par
\pard\f0{}}\par
\pard{}the operation will be optimised into a mere copy, and you can get {\f1{}&AE;} this way, if you had some in your input file.  But if you explicitly defeat the optimisation, like this maybe:\par
\par
{\pard\keep\li720\f1{}recode h3..u2,u2..h3 < {\i input}\par
\pard\f0{}}\par
\pard{}then {\f1{}&AE;} should be normalised into {\f1{}&AElig;} by the operation.\par
\par
\page\pard Node: {\b LaTeX}, \keepn Next: {\uldb Texinfo}{\v Texinfo}, Prev: {\uldb HTML}{\v HTML}, Up: {\uldb Miscellaneous}{\v Miscellaneous}\line
K{\footnote K LaTeX}
#{\footnote LaTeX}
${\footnote \pard{}LaTeX macro calls}\par
\pard{\fs24\b LaTeX macro calls}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}LaTeX} and has {\f1{}ltex} as an alias.  It is used for ASCII files coded to be read by LaTeX or, in certain cases, by TeX.\par
\par
\pard{}Whenever you recode from another charset to {\f1{}LaTeX}, beware that all occurrences of backslashes {\f1{}\'5C} are translated into the string {\f1{}\'5Cbackslash\'7B\'7D}.  However, in practice, people often use backslashes in the other charset for introducing TeX commands, compromising it: it is not pure TeX, nor it is pure other charset.  This translation of backslashes into {\f1{}\'5Cbackslash\'7B\'7D} can be rather inconvenient, it may be inhibited through the command option {\f1{}-d} (see {\uldb Mixed}{\v Mixed}).\par
\par
\page\pard Node: {\b Texinfo}, \keepn Next: {\uldb African}{\v African}, Prev: {\uldb LaTeX}{\v LaTeX}, Up: {\uldb Miscellaneous}{\v Miscellaneous}\line
K{\footnote K Texinfo}
#{\footnote Texinfo}
${\footnote \pard{}GNU project documentation files}\par
\pard{\fs24\b GNU project documentation files}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}Texinfo} and has {\f1{}texi} and {\f1{}ti} for aliases.  It is used by the GNU project for its documentation.  Texinfo files may be converted into Info files by the {\f1{}makeinfo} program and into nice printed manuals by the TeX system.\par
\par
\pard{}Even if {\f1{}recode} may transform other charsets to Texinfo, it may not read Texinfo files yet.  In these times, usages are also changing between versions of Texinfo, and {\f1{}recode} only partially succeeds in correctly following these changes.  So, for now, Texinfo support in {\f1{}recode} should be considered as work still in progress (!).\par
\par
\page\pard Node: {\b African}, \keepn Next: {\uldb Cyrillic}{\v Cyrillic}, Prev: {\uldb Texinfo}{\v Texinfo}, Up: {\uldb Miscellaneous}{\v Miscellaneous}\line
K{\footnote K African}
#{\footnote African}
${\footnote \pard{}African charsets}\par
\pard{\fs24\b African charsets}\par
\par
\pard{}Some African character sets are available for a few languages, when these are heavily used in countries where French is also currently spoken.\par
\par
\pard{}One African charset is usable for Bambara, Ewondo and Fulfude, as well as for French.  This charset is available in {\f1{}recode} under the name {\f1{}AFRFUL-102-BPI_OCIL}.  Accepted aliases are {\f1{}bambara}, {\f1{}bra}, {\f1{}ewondo} and {\f1{}fulfude}.  Transliterated forms of the same are available under the name {\f1{}AFRFUL-103-BPI_OCIL}.  Accepted aliases are {\f1{}t-bambara}, {\f1{}t-bra}, {\f1{}t-ewondo} and {\f1{}t-fulfude}.\par
\par
\pard{}Another African charset is usable for Lingala, Sango and Wolof, as well as for French.  This charset is available in {\f1{}recode} under the name {\f1{}AFRLIN-104-BPI_OCIL}.  Accepted aliases are {\f1{}lingala}, {\f1{}lin}, {\f1{}sango} and {\f1{}wolof}.  Transliterated forms of the same are available under the name {\f1{}AFRLIN-105-BPI_OCIL}.  Accepted aliases are {\f1{}t-lingala}, {\f1{}t-lin}, {\f1{}t-sango} and {\f1{}t-wolof}.\par
\par
\pard{}To ease exchange with {\f1{}ISO-8859-1}, there is a charset conveying transliterated forms for Latin-1 in a way which is compatible with the other African charsets in this series.  This charset is available in {\f1{}recode} under the name {\f1{}AFRL1-101-BPI_OCIL}.  Accepted aliases are {\f1{}t-fra} and {\f1{}t-francais}.\par
\par
\page\pard Node: {\b Cyrillic}, \keepn Next: {\uldb Texte}{\v Texte}, Prev: {\uldb African}{\v African}, Up: {\uldb Miscellaneous}{\v Miscellaneous}\line
K{\footnote K Cyrillic}
#{\footnote Cyrillic}
${\footnote \pard{}Cyrillic charsets}\par
\pard{\fs24\b Cyrillic charsets}\par
\par
\pard{}The following Cyrillic charsets are already available in {\f1{}recode} through RFC 1345 tables: {\f1{}CP1251} with aliases {\f1{}1251}, {\f1{} ms-cyrl} and {\f1{}windows-1251}; {\f1{}CSN_369103} with aliases {\f1{}ISO-IR-139} and {\f1{}KOI8_L2}; {\f1{}ECMA-cyrillic} with aliases {\f1{}ECMA-113}, {\f1{}ECMA-113:1986} and {\f1{}iso-ir-111}, {\f1{}IBM880} with aliases {\f1{}880}, {\f1{}CP880} and {\f1{}EBCDIC-Cyrillic}; {\f1{}INIS-cyrillic} with alias {\f1{}iso-ir-51}; {\f1{}ISO-8859-5} with aliases {\f1{}cyrillic}, {\f1{} ISO-8859-5:1988} and {\f1{}iso-ir-144}; {\f1{}KOI-7}; {\f1{}KOI-8} with alias {\f1{}GOST_19768-74}; {\f1{}KOI8-R}; {\f1{}KOI8-RU} and finally {\f1{}KOI8-U}.\par
\par
\pard{}There seems to remain some confusion in Cyrillic charsets, and because a few users requested it repeatedly, {\f1{}recode} now offers special services in that area.  Consider these charsets as experimental and debatable, as the extraneous tables describing them are still a bit fuzzy or non-standard.  Hopefully, in the long run, Cyrillic will be covered in Keld Simonsen's works to the satisfaction of everybody, and this section will merely disappear.\par
\par
{\pard{}{\f1{}KEYBCS2}\par
\pard\li720{}This charset is available under the name {\f1{}KEYBCS2}, with {\f1{}Kamenicky} as an accepted alias.\par
\par
\pard{}{\f1{}CORK}\par
\pard\li720{}This charset is available under the name {\f1{}CORK}, with {\f1{}T1} as an accepted alias.\par
\par
\pard{}{\f1{}KOI-8_CS2}\par
\pard\li720{}This charset is available under the name {\f1{}KOI-8_CS2}.\par
\par
\pard{}}\par
\page\pard Node: {\b Texte}, \keepn Next: {\uldb Mule}{\v Mule}, Prev: {\uldb Cyrillic}{\v Cyrillic}, Up: {\uldb Miscellaneous}{\v Miscellaneous}\line
K{\footnote K Texte}
#{\footnote Texte}
${\footnote \pard{}Easy French conventions}\par
\pard{\fs24\b Easy French conventions}\par
\par
\pard{}This charset is available in {\f1{}recode} under the name {\f1{}Texte} and has {\f1{}txte} for an alias.  It is a seven bits code, identical to {\f1{}ASCII-BS}, save for French diacritics which are noted using a slightly different convention.\par
\par
\pard{}At text entry time, these conventions provide a little speed up.  At read time, they slightly improve the readability over a few alternate ways of coding diacritics.  Of course, it would better to have a specialised keyboard to make direct eight bits entries and fonts for immediately displaying eight bit ISO Latin-1 characters.  But not everybody is so fortunate.  In a few mailing environments, and sadly enough, it still happens that the eight bit is often willing-fully destroyed.\par
\par
\pard{}Easy French has been in use in France for a while.  I only slightly adapted it (the diaeresis option) to make it more comfortable to several usages in Que'bec originating from Universite' de Montre'al.  In fact, the main problem for me was not to necessarily to invent Easy French, but to recognise the "best" convention to use, (best is not being defined, here) and to try to solve the main pitfalls associated with the selected convention.  Shortly said, we have:\par
\par
{\pard{}{\f1{}e'}\par
\pard\li720{}for {\f1{}e} (and some other vowels) with an acute accent,\par
\par
\pard{}{\f1{}e`}\par
\pard\li720{}for {\f1{}e} (and some other vowels) with a grave accent,\par
\par
\pard{}{\f1{}e^}\par
\pard\li720{}for {\f1{}e} (and some other vowels) with a circumflex accent,\par
\par
\pard{}{\f1{}e"}\par
\pard\li720{}for {\f1{}e} (and some other vowels) with a diaeresis,\par
\par
\pard{}{\f1{}c,}\par
\pard\li720{}for {\f1{}c} with a cedilla.\par
\pard{}}\par
\pard{}There is no attempt at expressing the {\f1{}ae} and {\f1{}oe} diphthongs.  French also uses tildes over {\f1{}n} and {\f1{}a}, but seldomly, and this is not represented either.  In some countries, {\f1{}:} is used instead of {\f1{}"} to mark diaeresis.  {\f1{}recode} supports only one convention per call, depending on the {\f1{}-c} option of the {\f1{}recode} command.  French quotes (sometimes called "angle quotes") are noted the same way English quotes are noted in TeX, {\i id est} by {\f1{}``} and {\f1{}''}.  No effort has been put to preserve Latin ligatures ({\f1{}ae}, {\f1{}oe}) which are representable in several other charsets.  So, these ligatures may be lost through Easy French conventions.\par
\par
\pard{}The convention is prone to losing information, because the diacritic meaning overloads some characters that already have other uses.  To alleviate this, some knowledge of the French language is boosted into the recognition routines.  So, the following subtleties are systematically obeyed by the various recognisers.\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}1. \tab{}A comma which follows a {\f1{}c} is interpreted as a cedilla only if it is followed by one of the vowels {\f1{}a}, {\f1{}o} or {\f1{}u}.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}2. \tab{}A single quote which follows a {\f1{}e} does not necessarily means an acute accent if it is followed by a single other one.  For example:\par
\par
{\pard\li720{}{\f1{}e'}\par
\pard\li1440{}will give an {\f1{}e} with an acute accent.\par
\par
\pard\li720{}{\f1{}e''}\par
\pard\li1440{}will give a simple {\f1{}e}, with a closing quotation mark.\par
\par
\pard\li720{}{\f1{}e'''}\par
\pard\li1440{}will give an {\f1{}e} with an acute accent, followed by a closing quotation mark.\par
\pard\li720{}}\par
\pard\li720{}There is a problem induced by this convention if there are English quotations with a French text.  In sentences like:\par
\par
{\pard\keep\li1440\f1{}There's a meeting at Archie's restaurant.\par
\pard\li720\f0{}}\par
\pard\li720{}the single quotes will be mistaken twice for acute accents.  So English contractions and suffix possessives could be mangled.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}3. \tab{}A double quote or colon, depending on {\f1{}-c} option, which follows a vowel is interpreted as diaeresis only if it is followed by another letter.  But there are in French several words that {\i end} with a diaeresis, and the {\f1{}recode} library is aware of them.  There are words ending in "igue", either feminine words without a relative masculine (besaigue" and cigue"), or feminine words with a relative masculine({\ul 1}{\v Texte_1}) (aigue", ambigue", contigue", exigue", subaigue" and suraigue").  There are also words not ending in "igue", but instead, either ending by "i"({\ul 2}{\v Texte_2}) {(ai", congai", goi", hai"kai", inoui", sai", samurai", thai" and tokai"), }ending by "e" (canoe") or ending by "u"({\ul 3}{\v Texte_3}) (Esau").\par
\par
\pard\li720{}Just to complete this topic, note that it would be wrong to make a rule for all words ending in "igue" as needing a diaerisis, as there are counter-examples (becfigue, be`sigue, bigue, bordigue, bourdigue, brigue, contre-digue, digue, d'intrigue, fatigue, figue, garrigue, gigue, igue, intrigue, ligue, prodigue, sarigue and zigue).\par
\par
\pard{}}\par
\page\pard#{\footnote Texte_1}
\pard{}(1) There are supposed to be seven words in this case.  So, one is missing.\par
\par
\page\pard#{\footnote Texte_2}
\pard{}(2) Look at one of the following sentences (the second has to be interpreted with the {\f1{}-c} option):\par
\par
{\pard\keep\li720\f1{}"Ai"e!  Voici le proble`me que j'ai"\line
Ai:e!  Voici le proble`me que j'ai:\par
\pard\f0{}}\par
\pard{}There is an ambiguity between an {ai", }the small animal, and the indicative future of {\i avoir} (first person singular), when followed by what could be a diaeresis mark.  Hopefully, the case is solved by the fact that an apostrophe always precedes the verb and almost never the animal.\par
\par
\page\pard#{\footnote Texte_3}
\pard{}(3) I did not pay attention to proper nouns, but this one showed up as being fairly evident.\par
\par
\page\pard Node: {\b Mule}, \keepn Next: {\uldb }{\v }, Prev: {\uldb Texte}{\v Texte}, Up: {\uldb Miscellaneous}{\v Miscellaneous}\line
K{\footnote K Mule}
#{\footnote Mule}
${\footnote \pard{}Mule as a multiplexed charset}\par
\pard{\fs24\b Mule as a multiplexed charset}\par
\par
\pard{}This version of {\f1{}recode} barely starts supporting multiplexed or super-charsets, that is, those encoding methods by which a single text stream may contain a combination of more than one constituent charset.  The only multiplexed charset in {\f1{}recode} is {\f1{}Mule}, and even then, it is only very partially implemented: the only correspondence available is with {\f1{}Latin-1}.  The author fastly implemented this only because he needed this for himself.  However, it is intended that Mule support to become more real in subsequent releases of {\f1{}recode}.\par
\par
\pard{}Multiplexed charsets are not to be confused with mixed charset texts (see {\uldb Mixed}{\v Mixed}).  For mixed charset input, the rules allowing to distinguish which charset is current, at any given place, are kind of informal, and driven from the semantics of what the file contains.  On the other side, multiplexed charsets are {\i designed} to be interpreted fairly precisely, and quite independently of any informational context.\par
\par
\pard{}The spelling {\f1{}Mule} originally stands for {\i {\i mul}tilingual {\i e}nhancement to GNU Emacs}, it is the result of a collective effort orchestrated by Handa Ken'ishi since 1993.  When {\f1{}Mule} got rewritten in the main development stream of GNU Emacs 20, the FSF renamed it {\f1{}MULE}, meaning {\i {\i mul}tilingual {\i e}nvironment in GNU Emacs}.  Even if the charset {\f1{}Mule} is meant to stay internal to GNU Emacs, it sometimes breaks loose in external files, and as a consequence, a recoding tool is sometimes needed.  Within Emacs, {\f1{}Mule} comes with {\f1{}leim}, which stands for {\i {\i l}ibraries of {\i e}macs {\i i}nput {\i m}ethods}.  One of these libraries is named {\f1{}quail}({\ul 1}{\v Mule_1}).\par
\par
\page\pard#{\footnote Mule_1}
\pard{}(1) Usually, quail means quail egg in Japanese, while egg alone is usually chicken egg.  Both quail egg and chicken egg are popular food in Japan.  The {\f1{}quail} input system has been named because it is smaller that the previous {\f1{}EGG} system.  As for {\f1{}EGG}, it is the translation of {\f1{}TAMAGO}.  This word comes from the Japanese sentence {\i {\i ta}kusan {\i ma}tasete {\i go}mennasai}, meaning {\i sorry to have let you wait so long}.  Of course, the publication of {\f1{}EGG} has been delayed many times...  (Story by Takahashi Naoto)\par
\par
\page\pard Node: {\b Surfaces}, \keepn Next: {\uldb Internals}{\v Internals}, Prev: {\uldb Miscellaneous}{\v Miscellaneous}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Surfaces}
#{\footnote Surfaces}
${\footnote \pard{}All about surfaces}\par
\pard{\fs24\b All about surfaces}\par
\par
\pard{}The \'A2trivial surface\'A2 consists into using a fixed number of bits (often eight) for each character, the bits together hold the integer value of the index for the character in its charset table.  There are many kinds of surfaces, beyond the trivial one, all having the purpose of increasing selected qualities for the storage or transmission.  For example, surfaces might increase the resistance to channel limits ({\f1{}Base64}), the transmission speed ({\f1{}gzip}), the information privacy ({\f1{}DES}), the conformance to operating system conventions ({\f1{}CR-LF}), the blocking into records ({\f1{}VB}), and surely other things as well({\ul 1}{\v Surfaces_1}).  Many surfaces may be applied to a stream of characters from a charset, the order of application of surfaces is important, and surfaces should be removed in the reverse order of their application.\par
\par
\pard{}Even if surfaces may generally be applied to various charsets, some surfaces were specifically designed for a particular charset, and would not make much sense if applied to other charsets.  In such cases, these conceptual surfaces have been implemented as {\f1{}recode} charsets, instead of as surfaces.  This choice yields to cleaner syntax and usage.  See {\uldb Universal}{\v Universal}.\par
\par
\pard{}Surfaces are implemented within {\f1{}recode} as special charsets which may only transform to or from the {\f1{}data} special charset.  Clever users may use this knowledge for writing surface names in requests exactly as if they were pure charsets, when the only need is to change surfaces without any kind of recoding between real charsets.  In such contexts, {\f1{}data} may also be used as if it were some kind of generic, anonymous charset: the request {\f1{}data..{\i surface}} merely adds the given {\i surface}, while the request {\f1{}{\i surface}..data} removes it.\par
\par
\pard{}We are only beginning to experiment with surfaces in {\f1{}recode}, but the concept opens the doors to many avenues, it is not clear yet which ones are worth pursuing, and which should be abandoned.  This chapter presents all surfaces currently available.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Permutations}{\v Permutations}\tab Permuting groups of bytes\par
{\uldb End lines}{\v End_lines}\tab Representation for end of lines\par
{\uldb MIME}{\v MIME}\tab MIME contents encodings\par
{\uldb Dump}{\v Dump}\tab Interpreted character dumps\par
{\uldb Debugging}{\v Debugging}\tab Artificial data\par
\pard{}}\par
\page\pard#{\footnote Surfaces_1}
\pard{}(1) These are mere examples to explain the concept, {\f1{}recode} only has {\f1{}Base64} and {\f1{}CR-LF}, actually.\par
\par
\page\pard Node: {\b Permutations}, \keepn Next: {\uldb End lines}{\v End_lines}, Prev: {\uldb Surfaces}{\v Surfaces}, Up: {\uldb Surfaces}{\v Surfaces}\line
K{\footnote K Permutations}
#{\footnote Permutations}
${\footnote \pard{}Permuting groups of bytes}\par
\pard{\fs24\b Permuting groups of bytes}\par
\par
\pard{}A permutation is a surface transformation which reorders groups of eight-bit bytes.  A {\i 21} permutation exchanges pairs of successive bytes.  If the text contains an odd number of bytes, the last byte is merely copied.  An {\i 4321} permutation inverts the order of quadruples of bytes.  If the text does not contains a multiple of four bytes, the remaining bytes are nevertheless permuted as {\i 321} if there are three bytes, {\i 21} if there are two bytes, or merely copied otherwise.\par
\par
{\pard{}{\f1{}21}\par
\pard\li720{}This surface is available in {\f1{}recode} under the name {\f1{}21-Permutation} and has {\f1{}swabytes} for an alias.\par
\par
\pard{}{\f1{}4321}\par
\pard\li720{}This surface is available in {\f1{}recode} under the name {\f1{}4321-Permutation}.\par
\par
\pard{}}\par
\page\pard Node: {\b End lines}, \keepn Next: {\uldb MIME}{\v MIME}, Prev: {\uldb Permutations}{\v Permutations}, Up: {\uldb Surfaces}{\v Surfaces}\line
K{\footnote K End lines}
#{\footnote End_lines}
${\footnote \pard{}Representation for end of lines}\par
\pard{\fs24\b Representation for end of lines}\par
\par
\pard{}The same charset might slightly differ, from one system to another, for the single fact that end of lines are not represented identically on all systems.  The representation for an end of line within {\f1{}recode} is the {\f1{}ASCII} or {\f1{}UCS} code with value 10, or {\f1{}LF}.  Other conventions for representing end of lines are available through surfaces.\par
\par
{\pard{}{\f1{}CR}\par
\pard\li720{}This convention is popular on Apple's Macintosh machines.  When this surface is applied, each line is terminated by {\f1{}CR}, which has {\f1{}ASCII} value 13.  Unless the library is operating in strict mode, adding or removing the surface will in fact {\i exchange} {\f1{}CR} and {\f1{}LF}, for better reversibility.  However, in strict mode, the exchange does not happen, any {\f1{}CR} will be copied verbatim while applying the surface, and any {\f1{}LF} will be copied verbatim while removing it.\par
\par
\pard\li720{}This surface is available in {\f1{}recode} under the name {\f1{}CR}, it does not have any aliases.  This is the implied surface for the Apple Macintosh related charsets.\par
\par
\pard{}{\f1{}CR-LF}\par
\pard\li720{}This convention is popular on Microsoft systems running on IBM PCs and compatible.  When this surface is applied, each line is terminated by a sequence of two characters: one {\f1{}CR} followed by one {\f1{}LF}, in that order.\par
\par
\pard\li720{}For compatibility with oldish MS-DOS systems, removing a {\f1{}CR-LF} surface will discard the first encountered {\f1{}C-z}, which has {\f1{}ASCII} value 26, and everything following it in the text.  Adding this surface will not, however, append a {\f1{}C-z} to the result.\par
\par
\pard\li720{}This surface is available in {\f1{}recode} under the name {\f1{}CR-LF} and has {\f1{}cl} for an alias.  This is the implied surface for the IBM or Microsoft related charsets or code pages.\par
\par
\pard{}}\par
\pard{}Some other charsets might have their own representation for an end of line, which is different from {\f1{}LF}.  For example, this is the case of various {\f1{}EBCDIC} charsets, or {\f1{}Icon-QNX}.  The recoding of end of lines is intimately tied into such charsets, it is not available separately as surfaces.\par
\par
\page\pard Node: {\b MIME}, \keepn Next: {\uldb Dump}{\v Dump}, Prev: {\uldb End lines}{\v End_lines}, Up: {\uldb Surfaces}{\v Surfaces}\line
K{\footnote K MIME}
#{\footnote MIME}
${\footnote \pard{}MIME contents encodings}\par
\pard{\fs24\b MIME contents encodings}\par
\par
\pard{}RFC 1521 defines two 7-bit surfaces, meant to prepare 8-bit messages for transmission.  Base64 is especially usable for binary entities, while Quoted-Printable is especially usable for text entities, in those case the lower 128 characters of the underlying charset coincide with ASCII.\par
\par
{\pard{}{\f1{}Base64}\par
\pard\li720{}This surface is available in {\f1{}recode} under the name {\f1{}Base64}, with {\f1{}b64} and {\f1{}64} as acceptable aliases.\par
\par
\pard{}{\f1{}Quoted-Printable}\par
\pard\li720{}This surface is available in {\f1{}recode} under the name {\f1{}Quoted-Printable}, with {\f1{}quote-printable} and {\f1{}QP} as acceptable aliases.\par
\par
\pard{}}\par
\pard{}Note that {\f1{}UTF-7}, which may be also considered as a MIME surface, is provided as a genuine charset instead, as it necessary relates to {\f1{}UCS-2} and nothing else.  See {\uldb UTF-7}{\v UTF_7}.\par
\par
\pard{}A little historical note, also showing the three levels of acceptation of Internet standards.  MIME changed from a "Proposed Standard" to a "Draft Standard" in 1993, and only became a "Full Standard" during 1996-12.\par
\par
\page\pard Node: {\b Dump}, \keepn Next: {\uldb Debugging}{\v Debugging}, Prev: {\uldb MIME}{\v MIME}, Up: {\uldb Surfaces}{\v Surfaces}\line
K{\footnote K Dump}
#{\footnote Dump}
${\footnote \pard{}Interpreted character dumps}\par
\pard{\fs24\b Interpreted character dumps}\par
\par
\pard{}Dumps are surfaces meant to express, in ways which are a bit more readable, the bit patterns used to represent characters.  They allow the inspection or debugging of character streams, but also, they may assist a bit the production of C source code which, once compiled, would hold in memory a copy of the original coding.  However, {\f1{}recode} does not attempt, in any way, to produce complete C source files in dumps.  User hand editing or {\f1{}Makefile} trickery is still needed for adding missing lines.  Dumps may be given in decimal, hexadecimal and octal, and be based over chunks of either one, two or four eight-bit bytes.  Formatting has been chosen to respect the C language syntax for number constants, with commas and newlines inserted appropriately.\par
\par
\pard{}However, when dumping two or four byte chunks, the last chunk may be incomplete.  This is observable through the usage of narrower expression for that last chunk only.  Such a shorter chunk would not be compiled properly within a C initialiser, as all members of an array share a single type, and so, have identical sizes.\par
\par
{\pard{}{\f1{}Octal-1}\par
\pard\li720{}This surface corresponds to an octal expression of each input byte.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Octal-1}, with {\f1{}o1} and {\f1{}o} as acceptable aliases.\par
\par
\pard{}{\f1{}Octal-2}\par
\pard\li720{}This surface corresponds to an octal expression of each pair of input bytes, except for the last pair, which may be short.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Octal-2} and has {\f1{}o2} for an alias.\par
\par
\pard{}{\f1{}Octal-4}\par
\pard\li720{}This surface corresponds to an octal expression of each quadruple of input bytes, except for the last quadruple, which may be short.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Octal-4} and has {\f1{}o4} for an alias.\par
\par
\pard{}{\f1{}Decimal-1}\par
\pard\li720{}This surface corresponds to an decimal expression of each input byte.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Decimal-1}, with {\f1{}d1} and {\f1{}d} as acceptable aliases.\par
\par
\pard{}{\f1{}Decimal-2}\par
\pard\li720{}This surface corresponds to an decimal expression of each pair of input bytes, except for the last pair, which may be short.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Decimal-2} and has {\f1{}d2} for an alias.\par
\par
\pard{}{\f1{}Decimal-4}\par
\pard\li720{}This surface corresponds to an decimal expression of each quadruple of input bytes, except for the last quadruple, which may be short.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Decimal-4} and has {\f1{}d4} for an alias.\par
\par
\pard{}{\f1{}Hexadecimal-1}\par
\pard\li720{}This surface corresponds to an hexadecimal expression of each input byte.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Hexadecimal-1}, with {\f1{}x1} and {\f1{}x} as acceptable aliases.\par
\par
\pard{}{\f1{}Hexadecimal-2}\par
\pard\li720{}This surface corresponds to an hexadecimal expression of each pair of input bytes, except for the last pair, which may be short.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Hexadecimal-2}, with {\f1{}x2} for an alias.\par
\par
\pard{}{\f1{}Hexadecimal-4}\par
\pard\li720{}This surface corresponds to an hexadecimal expression of each quadruple of input bytes, except for the last quadruple, which may be short.\par
\par
\pard\li720{}It is available in {\f1{}recode} under the name {\f1{}Hexadecimal-4}, with {\f1{}x4} for an alias.\par
\par
\pard{}}\par
\pard{}When removing a dump surface, that is, when reading a dump results back into a sequence of bytes, the narrower expression for a short last chunk is recognised, so dumping is a fully reversible operation.  However, in case you want do produce dumps by other means than through {\f1{}recode}, beware that for decimal dumps, the library has to rely on the number of spaces to establish the original byte size of the chunk.\par
\par
\pard{}Despite the library might report reversibility errors, removing a dump surface is a rather forgiving process: one may mix bases, group more or less numbers per source line, or use shorter chunks elsewhere than at the far end.  Also, source lines not beginning with a number are skipped.  So, {\f1{}recode} should often be able to read a whole C header file, wrapping the results of a previous dump, and regenerate the original byte string.\par
\par
\page\pard Node: {\b Debugging}, \keepn Next: {\uldb }{\v }, Prev: {\uldb Dump}{\v Dump}, Up: {\uldb Surfaces}{\v Surfaces}\line
K{\footnote K Debugging}
#{\footnote Debugging}
${\footnote \pard{}Artificial data}\par
\pard{\fs24\b Artificial data}\par
\par
\pard{}A few pseudo-surfaces exist to generate debugging data out of thin air.  These surfaces are only meant for the expert {\f1{}recode} user, and are only useful in a few contexts, like for generating binary permutations from the recoding or acting on them.\par
\par
\pard{}Debugging surfaces, {\i when removed}, insert their generated data at the beginning of the output stream, and copy all the input stream after the generated data, unchanged.  This strange removal constraint comes from the fact that debugging surfaces are usually specified in the {\i before} position instead of the {\i after} position within a request.  With debugging surfaces, one often recodes file {\f1{}/dev/null} in filter mode.  Specifying many debugging surfaces at once has an accumulation effect on the output, and since surfaces are removed from right to left, each generating its data at the beginning of previous output, the net effect is an {\i impression} that debugging surfaces are generated from left to right, each appending to the result of the previous.  In any case, any real input data gets appended after what was generated.\par
\par
{\pard{}{\f1{}test7}\par
\pard\li720{}When removed, this surface produces 128 single bytes, the first having value 0, the second having value 1, and so forth until all 128 values have been generated.\par
\par
\pard{}{\f1{}test8}\par
\pard\li720{}When removed, this surface produces 256 single bytes, the first having value 0, the second having value 1, and so forth until all 256 values have been generated.\par
\par
\pard{}{\f1{}test15}\par
\pard\li720{}When removed, this surface produces 64509 double bytes, the first having value 0, the second having value 1, and so forth until all values have been generated, but excluding risky {\f1{}UCS-2} values, like all codes from the surrogate {\f1{}UCS-2} area (for {\f1{}UTF-16}), the byte order mark, and values known as invalid {\f1{}UCS-2}.\par
\par
\pard{}{\f1{}test16}\par
\pard\li720{}When removed, this surface produces 65536 double bytes, the first having value 0, the second having value 1, and so forth until all 65536 values have been generated.\par
\par
\pard{}}\par
\pard{}For an example, the command {\f1{}recode l5/test8..dump < /dev/null} is a convoluted way to produce an output similar to {\f1{}recode -lf l5}.  It says to generate all possible 256 bytes and interpret them as {\f1{}ISO-8859-9} codes, while converting them to {\f1{}UCS-2}.  Resulting {\f1{}UCS-2} characters are dumped one per line, accompanied with their explicative name.\par
\par
\page\pard Node: {\b Internals}, \keepn Next: {\uldb }{\v }, Prev: {\uldb Surfaces}{\v Surfaces}, Up: {\uldb Top}{\v Top}\line
K{\footnote K Internals}
#{\footnote Internals}
${\footnote \pard{}Internal aspects}\par
\pard{\fs24\b Internal aspects}\par
\par
\pard{}The incoming explanations of the internals of {\f1{}recode} should help people who want to dive into {\f1{}recode} sources for adding new charsets.  Adding new charsets does not require much knowledge about the overall organisation of {\f1{}recode}.  You can rather concentrate of your new charset, letting the remainder of the {\f1{}recode} mechanics take care of interconnecting it with all others charsets.\par
\par
\pard{}If you intend to play seriously at modifying {\f1{}recode}, beware that you may need some other GNU tools which were not required when you first installing {\f1{}recode}.  If you modify or create any {\f1{}.l} file, then you need Flex, and some better {\f1{}awk} like {\f1{}mawk}, GNU {\f1{}awk}, or {\f1{}nawk}.  If you modify the documentation (and you should!), you need {\f1{}makeinfo}.  If you are really audacious, you may also want Perl for modifying tabular processing, then {\f1{}m4}, Autoconf, Automake and {\f1{}libtool} for adjusting configuration matters.\par
\par
{\par
\pard\keep\fi-2880\li2880\tx2880{}{\uldb Main flow}{\v Main_flow}\tab Overall organisation\par
{\uldb New charsets}{\v New_charsets}\tab Adding new charsets\par
{\uldb New surfaces}{\v New_surfaces}\tab Adding new surfaces\par
{\uldb Design}{\v Design}\tab Comments on the library design\par
\pard{}}\par
\page\pard Node: {\b Main flow}, \keepn Next: {\uldb New charsets}{\v New_charsets}, Prev: {\uldb Internals}{\v Internals}, Up: {\uldb Internals}{\v Internals}\line
K{\footnote K Main flow}
#{\footnote Main_flow}
${\footnote \pard{}Overall organisation}\par
\pard{\fs24\b Overall organisation}\par
\par
\pard{}The {\f1{}recode} mechanics slowly evolved for many years, and it would be tedious to explain all problems I met and mistakes I did all along, yielding the current behaviour.  Surely, one of the key choices was to stop trying to do all conversions in memory, one line or one buffer at a time.  It has been fruitful to use the character stream paradigm, and the elementary recoding steps now convert a whole stream to another.  Most of the control complexity in {\f1{}recode} exists so that each elementary recoding step stays simple, making easier to add new ones.  The whole point of {\f1{}recode}, as I see it, is providing a comfortable nest for growing new charset conversions.\par
\par
\pard{}The main {\f1{}recode} driver constructs, while initialising all conversion modules, a table giving all the conversion routines available (\'A2single step\'A2s) and for each, the starting charset and the ending charset.  If we consider these charsets as being the nodes of a directed graph, each single step may be considered as oriented arc from one node to the other.  A cost is attributed to each arc: for example, a high penalty is given to single steps which are prone to losing characters, a lower penalty is given to those which need studying more than one input character for producing an output character, etc.\par
\par
\pard{}Given a starting code and a goal code, {\f1{}recode} computes the most economical route through the elementary recodings, that is, the best sequence of conversions that will transform the input charset into the final charset.  To speed up execution, {\f1{}recode} looks for subsequences of conversions which are simple enough to be merged, it then dynamically creates new single steps to represent these mergings.\par
\par
\pard{}A \'A2double step\'A2 in {\f1{}recode} is a special concept representing a sequence of two single steps, the output of the first single step being the special charset {\f1{}UCS-2}, the input of the second single step being also {\f1{}UCS-2}.  Special {\f1{}recode} machinery dynamically produces efficient, reversible, merge-able single steps out of these double steps.\par
\par
\pard{}I made some statistics about how many internal recoding steps are required between any two charsets chosen at random.  The initial recoding layout, before optimisation, always uses between 1 and 6 steps.  Optimisation could sometimes produce mere copies, which are counted as no steps at all.  In other cases, optimisation is unable to save any step.  The number of steps after optimisation is currently between 0 and 6 steps.  Of course, the {\i expected} number of steps is affected by optimisation: it drops from 2.2 to 0.9.({\ul 1}{\v Main_flow_1}) This means that {\f1{}recode} uses a theoretical average of a bit less than one step per recoding job.  This looks good.  This was computed using reversible recodings.  In strict mode, optimisation might be defeated somewhat.  Number of steps run between 1 and 6, both before and after optimisation, and the expected number of steps decreases by a lesser amount, going from 2.2 to 1.3.  This is still manageable.\par
\par
\page\pard#{\footnote Main_flow_1}
\pard{}(1) The above numbers were for {\f1{}recode} version 3.4.  After UCS additions, it drops from 2.3 to 1.1.\par
\par
\page\pard Node: {\b New charsets}, \keepn Next: {\uldb New surfaces}{\v New_surfaces}, Prev: {\uldb Main flow}{\v Main_flow}, Up: {\uldb Internals}{\v Internals}\line
K{\footnote K New charsets}
#{\footnote New_charsets}
${\footnote \pard{}Adding new charsets}\par
\pard{\fs24\b Adding new charsets}\par
\par
\pard{}The main part of {\f1{}recode} is written in C, as are most single steps.  A few single steps need to recognise sequences of multiple characters, they are often better written in Flex.  It is easy for a programmer to add a new charset to {\f1{}recode}.  All it requires is making a few functions kept in a single {\f1{}.c} file, adjusting {\f1{}Makefile.am} and remaking {\f1{}recode}.\par
\par
\pard{}One of the function should convert from any previous charset to the new one.  Any previous charset will do, but try to select it so you will not lose too much information while converting.  The other function should convert from the new charset to any older one.  You do not have to select the same old charset than what you selected for the previous routine.  Once again, select any charset for which you will not lose too much information while converting.\par
\par
\pard{}If, for any of these two functions, you have to read multiple bytes of the old charset before recognising the character to produce, you might prefer programming it in Flex in a separate {\f1{}.l} file.  Prototype your C or Flex files after one of those which exist already, so to keep the sources uniform.  Besides, at {\f1{}make} time, all {\f1{}.l} files are automatically merged into a single big one by the script {\f1{}mergelex.awk}.\par
\par
\pard{}There are a few hidden rules about how to write new {\f1{}recode} modules, for allowing the automatic creation of {\f1{}decsteps.h} and {\f1{}initsteps.h} at {\f1{}make} time, or the proper merging of all Flex files.  Mimetism is a simple approach which relieves me of explaining all these rules!  Start with a module closely resembling what you intend to do.  Here is some advice for picking up a model.  First decide if your new charset module is to be be driven by algorithms rather than by tables.  For algorithmic recodings, see {\f1{}iconqnx.c} for C code, or {\f1{}txtelat1.l} for Flex code.  For table driven recodings, see {\f1{}ebcdic.c} for one-to-one style recodings, {\f1{}lat1html.c} for one-to-many style recodings, or {\f1{}atarist.c} for double-step style recodings.  Just select an example from the style that better fits your application.\par
\par
\pard{}Each of your source files should have its own initialisation function, named {\f1{}module_{\i charset}}, which is meant to be executed {\i quickly} once, prior to any recoding.  It should declare the name of your charsets and the single steps (or elementary recodings) you provide, by calling {\f1{}declare_step} one or more times.  Besides the charset names, {\f1{}declare_step} expects a description of the recoding quality (see {\f1{}recodext.h}) and two functions you also provide.\par
\par
\pard{}The first such function has the purpose of allocating structures, pre-conditioning conversion tables, etc.  It is also the way of further modifying the {\f1{}STEP} structure.  This function is executed if and only if the single step is retained in an actual recoding sequence.  If you do not need such delayed initialisation, merely use {\f1{}NULL} for the function argument.\par
\par
\pard{}The second function executes the elementary recoding on a whole file.  There are a few cases when you can spare writing this function:\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Some single steps do nothing else than a pure copy of the input onto the output, in this case, you can use the predefined function {\f1{}file_one_to_one}, while having a delayed initialisation for presetting the {\f1{}STEP} field {\f1{}one_to_one} to the predefined value {\f1{}one_to_same}.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Some single steps are driven by a table which recodes one character into another; if the recoding does nothing else, you can use the predefined function {\f1{}file_one_to_one}, while having a delayed initialisation for presetting the {\f1{}STEP} field {\f1{}one_to_one} with your table.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Some single steps are driven by a table which recodes one character into a string; if the recoding does nothing else, you can use the predefined function {\f1{}file_one_to_many}, while having a delayed initialisation for presetting the {\f1{}STEP} field {\f1{}one_to_many} with your table.\par
\par
\pard{}}\par
\pard{}If you have a recoding table handy in a suitable format but do not use one of the predefined recoding functions, it is still a good idea to use a delayed initialisation to save it anyway, because {\f1{}recode} option {\f1{}-h} will take advantage of this information when available.\par
\par
\pard{}Finally, edit {\f1{}Makefile.am} to add the source file name of your routines to the {\f1{}C_STEPS} or {\f1{}L_STEPS} macro definition, depending on the fact your routines is written in C or in Flex.\par
\par
\page\pard Node: {\b New surfaces}, \keepn Next: {\uldb Design}{\v Design}, Prev: {\uldb New charsets}{\v New_charsets}, Up: {\uldb Internals}{\v Internals}\line
K{\footnote K New surfaces}
#{\footnote New_surfaces}
${\footnote \pard{}Adding new surfaces}\par
\pard{\fs24\b Adding new surfaces}\par
\par
\pard{}Adding a new surface is technically quite similar to adding a new charset.  See {\uldb New charsets}{\v New_charsets}.  A surface is provided as a set of two transformations: one from the predefined special charset {\f1{}data} to the new surface, meant to apply the surface, the other from the new surface to the predefined special charset {\f1{}data}, meant to remove the surface.\par
\par
\pard{}Internally in {\f1{}recode}, function {\f1{}declare_step} especially recognises when a charset is so related to {\f1{}data}, and then takes appropriate actions so that charset gets indeed installed as a surface.\par
\par
\page\pard Node: {\b Design}, \keepn Next: {\uldb }{\v }, Prev: {\uldb New surfaces}{\v New_surfaces}, Up: {\uldb Internals}{\v Internals}\line
K{\footnote K Design}
#{\footnote Design}
${\footnote \pard{}Comments on the library design}\par
\pard{\fs24\b Comments on the library design}\par
\par
{\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Why a shared library?\par
\par
\pard\li720{}There are many different approaches to reduce system requirements to handle all tables needed in the {\f1{}recode} library.  One of them is to have the tables in an external format and only read them in on demand.  After having pondered this for a while, I finally decided against it, mainly because it involves its own kind of installation complexity, and it is not clear to me that it would be as interesting as I first imagined.\par
\par
\pard\li720{}It looks more efficient to see all tables and algorithms already mapped into virtual memory from the start of the execution, yet not loaded in actual memory, than to go through many disk accesses for opening various data files once the program is already started, as this would be needed with other solutions.  Using a shared library also has the indirect effect of making various algorithms handily available, right in the same modules providing the tables.  This alleviates much the burden of the maintenance.\par
\par
\pard\li720{}Of course, I would like to later make an exception for only a few tables, built locally by users for their own particular needs once {\f1{}recode} is installed.  {\f1{}recode} should just go and fetch them.  But I do not perceive this as very urgent, yet useful enough to be worth implementing.\par
\par
\pard\li720{}Currently, all tables needed for recoding are precompiled into binaries, and all these binaries are then made into a shared library.  As an initial step, I turned {\f1{}recode} into a main program and a non-shared library, this allowed me to tidy up the API, get rid of all global variables, etc.  It required a surprising amount of program source massaging.  But once this cleaned enough, it was easy to use Gordon Matzigkeit's {\f1{}libtool} package, and take advantage of the Automake interface to neatly turn the non-shared library into a shared one.\par
\par
\pard\li720{}Sites linking with the {\f1{}recode} library, while their system does not support any form of shared libraries, might end up with bulky executables.  Surely, the {\f1{}recode} library will have to be used statically, and might not very nicely usable on such systems.  It seems that progress has a price for those being slow at it.\par
\par
\pard\li720{}There is a locality problem I did not address yet.  Currently, the {\f1{}recode} library takes many cycles to initialise itself, calling each module in turn for it to set up associated knowledge about charsets, aliases, elementary steps, recoding weights, etc.  {\i Then}, the recoding sequence is decided out of the command given.  I would not be surprised if initialisation was taking a perceivable fraction of a second on slower machines.  One thing to do, most probably not right in version 3.5, but the version after, would have {\f1{}recode} to pre-load all tables and dump them at installation time.  The result would then be compiled and added to the library.  This would spare many initialisation cycles, but more importantly, would avoid calling all library modules, scattered through the virtual memory, and so, possibly causing many spurious page exceptions each time the initialisation is requested, at least once per program execution.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Why not a central charset?\par
\par
\pard\li720{}It would be simpler, and I would like, if something like ISO 10646 was used as a turning template for all charsets in {\f1{}recode}.  Even if I think it could help to a certain extent, I'm still not fully sure it would be sufficient in all cases.  Moreover, some people disagree about using ISO 10646 as the central charset, to the point I cannot totally ignore them, and surely, {\f1{}recode} is not a mean for me to force my own opinions on people.  I would like that {\f1{}recode} be practical more than dogmatic, and reflect usage more than religions.\par
\par
\pard\li720{}Currently, if you ask {\f1{}recode} to go from {\i charset1} to {\i charset2} chosen at random, it is highly probable that the best path will be quickly found as:\par
\par
{{\i \pard\li1440\f1{}charset1}..{\f1{}UCS-2}..{\i charset2}\par
\pard\li720\f0{}}\par
\pard\li720{}That is, it will almost always use the {\f1{}UCS} as a trampoline between charsets.  However, {\f1{}UCS-2} will be immediately be optimised out, and {\i charset1}..{\i charset2} will often be performed in a single step through a permutation table generated on the fly for the circumstance ({\ul 1}{\v Design_1}).\par
\par
\pard\li720{}In those few cases where {\f1{}UCS-2} is not selected as a conceptual intermediate, I plan to study if it could be made so.  But I guess some cases will remain where {\f1{}UCS-2} is not a proper choice.  Even if {\f1{}UCS} is often the good choice, I do not intend to forcefully restrain {\f1{}recode} around {\f1{}UCS-2} (nor {\f1{}UCS-4}) for now.  We might come to that one day, but it will come out of the natural evolution of {\f1{}recode}.  It will then reflect a fact, rather than a preset dogma.\par
\par
\pard\fi-720\li720\tx144\tx720{}\tab{}{\f2\'B7}\tab{}Why not {\f1{}iconv}?\par
\par
\pard\li720{}The {\f1{}iconv} routine and library allows for converting characters from an input buffer to an input buffer, synchronously advancing both buffer cursors.  If the output buffer is not big enough to receive all of the conversion, the routine returns with the input cursor set at the position where the conversion could later be resumed, and the output cursor set to indicate until where the output buffer has been filled.  Despite this scheme is simple and nice, the {\f1{}recode} library does not offer it currently.  Why not?\par
\par
\pard\li720{}When long sequences of decodings, stepwise recodings, and re-encodings are involved, as it happens in true life, synchronising the input buffer back to where it should have stopped, when the output buffer becomes full, is a difficult problem.  Oh, we could make it simpler at the expense of loosing space or speed: by inserting markers between each input character and counting them at the output end; by processing only one character in a time through the whole sequence; by repeatedly attempting to recode various subsets of the input buffer, binary searching on their length until the output just fits.  The overhead of such solutions looks fully prohibitive to me, and the gain very minimal.  I do not see a real advantage, nowadays, imposing a fixed length to an output buffer.  It makes things so much simpler and efficient to just let the output buffer size float a bit.\par
\par
\pard\li720{}Of course, if the above problem was solved, the {\f1{}iconv} library should be easily emulated, given that {\f1{}recode} has similar knowledge about charsets, of course.  This either solved or not, the {\f1{}iconv} program remains trivial (given similar knowledge about charsets).  I also presume that the {\f1{}genxlt} program would be easy too, but I do not have enough detailed specifications of it to be sure.\par
\par
\pard\li720{}A lot of years ago, {\f1{}recode} was using a similar scheme, and I found it rather hard to manage for some cases.  I rethought the overall structure of {\f1{}recode} for getting away from that scheme, and never regretted it.  I perceive {\f1{}iconv} as an artificial solution which surely has some elegances and virtues, but I do not find it really useful as it stands: one always has to wrap {\f1{}iconv} into something more refined, extending it for real cases.  From past experience, I think it is unduly hard to fully implement this scheme.  It would be awkward that we do contortions for the sole purpose of implementing exactly its specification, without real, fairly sounded reasons (other then the fact some people once thought it was worth standardising).  It is much better to immediately aim for the refinement we need, without uselessly forcing us into the dubious detour {\f1{}iconv} represents.\par
\par
\pard\li720{}Some may argue that if {\f1{}recode} was using a comprehensive charset as a turning template, as discussed in a previous point, this would make {\f1{}iconv} easier to implement.  Some may be tempted to say that the cases which are hard to handle are not really needed, nor interesting, anyway.  I feel and fear a bit some pressure wanting that {\f1{}recode} be split into the part that well fits the {\f1{}iconv} model, and the part that does not fit, considering this second part less important, with the idea of dropping it one of these days, maybe.  My guess is that users of the {\f1{}recode} library, whatever its form, would not like to have such arbitrary limitations.  In the long run, we should not have to explain to our users that some recodings may not be made available just because they do not fit the simple model we had in mind when we did it.  Instead, we should try to stay opened to the difficulties of real life.  There is still a lot of complex needs for Asian people, say, that {\f1{}recode} does not currently address, while it should.  Not only the doors should stay opened, but we should force them wider!\par
\par
\pard{}}\par
\page\pard#{\footnote Design_1}
\pard{}(1) If strict mapping is requested, another efficient device will be used instead of a permutation.\par
\par
\page K{\footnote Makertf}#{\footnote hcMakertf}${\footnote \pard{}About Makertf}\pard{\fs24\b About Makertf}\par\par\pard{}Makertf is a program that converts "Texinfo" files into "Rich Text Format" (RTF) files. It can be used to make WinHelp Files from GNU manuals and other documentation written in Texinfo. Visit http://www.snafu.de/~cschenk/makertf for more information.}